0000463: Investigate and document methods of properly syncing mirrors.

ID	Project	Category	View Status	Date Submitted	Last Update

0000463	apt-panopticon	General	public	2023-01-11 22:57	2023-10-01 02:29

Reporter	onefang	Assigned To	onefang
Priority	high	Severity	feature	Reproducibility	N/A
Status	assigned	Resolution	open

Summary	0000463: Investigate and document methods of properly syncing mirrors.
Description	In #devuan on 2023-01-12 rwp mentioned an issue with package mirrors HTTP headers having a TTL that was screwing with apt-cacher-ng, causing it to fetch things that had not really been updated. "I never saw these types of apt-cacher-ng failures with Debian mirrors that I relatively routinely see with Devuan mirrors." "And I think that is because Devuan mirrors are updated in place with rsync but Debian mirrors are updated using the custom script ftpsync(sp?)."
Additional Information	Used to be "Investingate ftpsync, or whatever it is called." but I'm making this more generic.
Tags	No tags attached.

onefang 2023-10-01 02:24 administrator ~0000618	Come up with a more robust update scheme. rsync add all the new .deb files. rsync update all the metadata files. rsync delete old .deb files. Only there's some more nitty gritty devil in the details in the middle of that which should be tweaked for even more robustness. Especially for the metadata. Apparently ftpsync does that, and is used by Debian mirrors.

onefang 2023-10-01 02:27 administrator ~0000619 Last edited: 2023-10-01 02:28	According to Bob Proulx - " The debmirror script can be used to update mirrors "correctly" such that they are consistent before, during, and as they updated. I have found it light weight and accessible. That's one possible tool. The official Debian mirrors require use of the official ftpmirror script. That's probably the way Devuan should go too. https://www.debian.org/mirror/ftpmirror#how "

Date Modified	Username	Field	Change
2023-01-11 22:57	onefang	New Issue
2023-01-11 22:57	onefang	Status	new => assigned
2023-01-11 22:57	onefang	Assigned To	=> onefang
2023-10-01 02:19	onefang	Relationship added	child of 0000077
2023-10-01 02:23	onefang	Summary	Investingate ftpsync, or whatever it is called. => Investigate and document methods of properly syncing mirrors.
2023-10-01 02:23	onefang	Additional Information Updated
2023-10-01 02:24	onefang	Note Added: 0000618
2023-10-01 02:24	onefang	Relationship deleted	child of 0000077
2023-10-01 02:27	onefang	Note Added: 0000619
2023-10-01 02:28	onefang	Note Edited: 0000619
2023-10-01 02:28	onefang	Note Edited: 0000619
2023-10-01 02:29	onefang	Priority	low => high