View Issue Details

IDProjectCategoryView StatusLast Update
0000463apt-panopticonGeneralpublic2023-10-01 02:29
Reporteronefang Assigned Toonefang  
Status assignedResolutionopen 
Summary0000463: Investigate and document methods of properly syncing mirrors.
DescriptionIn #devuan on 2023-01-12 rwp mentioned an issue with package mirrors HTTP headers having a TTL that was screwing with apt-cacher-ng, causing it to fetch things that had not really been updated.

"I never saw these types of apt-cacher-ng failures with Debian mirrors that I relatively routinely see with Devuan mirrors."
"And I think that is because Devuan mirrors are updated in place with rsync but Debian mirrors are updated using the custom script ftpsync(sp?)."
Additional InformationUsed to be "Investingate ftpsync, or whatever it is called." but I'm making this more generic.
TagsNo tags attached.




2023-10-01 02:24

administrator   ~0000618

Come up with a more robust update scheme.

rsync add all the new .deb files.
rsync update all the metadata files.
rsync delete old .deb files.

Only there's some more nitty gritty devil in the details in the middle of that which should be tweaked for even more robustness. Especially for the metadata.

Apparently ftpsync does that, and is used by Debian mirrors.


2023-10-01 02:27

administrator   ~0000619

Last edited: 2023-10-01 02:28

According to Bob Proulx -

The debmirror script can be used to update mirrors "correctly" such that they are consistent before, during, and as they updated. I have found it light weight and accessible. That's one possible tool.

The official Debian mirrors require use of the official ftpmirror script. That's probably the way Devuan should go too.

Issue History

Date Modified Username Field Change
2023-01-11 22:57 onefang New Issue
2023-01-11 22:57 onefang Status new => assigned
2023-01-11 22:57 onefang Assigned To => onefang
2023-10-01 02:19 onefang Relationship added child of 0000077
2023-10-01 02:23 onefang Summary Investingate ftpsync, or whatever it is called. => Investigate and document methods of properly syncing mirrors.
2023-10-01 02:23 onefang Additional Information Updated
2023-10-01 02:24 onefang Note Added: 0000618
2023-10-01 02:24 onefang Relationship deleted child of 0000077
2023-10-01 02:27 onefang Note Added: 0000619
2023-10-01 02:28 onefang Note Edited: 0000619
2023-10-01 02:28 onefang Note Edited: 0000619
2023-10-01 02:29 onefang Priority low => high