aboutsummaryrefslogtreecommitdiffstatshomepage
diff options
context:
space:
mode:
Diffstat
-rw-r--r--README.md2
-rw-r--r--Report-web_0.html4
-rw-r--r--Report-web_1.html38
-rw-r--r--Report-web_2.html5
-rw-r--r--Report-web_3.html11
-rw-r--r--Report-web_TABLE.html158
-rwxr-xr-xapt-panopticon-report-email-web.lua97
-rwxr-xr-xapt-panopticon.lua1
-rw-r--r--explanations.html96
9 files changed, 255 insertions, 157 deletions
diff --git a/README.md b/README.md
index 982a108..9ff79f2 100644
--- a/README.md
+++ b/README.md
@@ -158,8 +158,6 @@ redirections. Unless a specific mirror is given on the command line, the
158mirror_list.txt file from pkgmaster.devuan.org is used to select mirrors 158mirror_list.txt file from pkgmaster.devuan.org is used to select mirrors
159to test. 159to test.
160 160
161explanations.html explains the tests in more detail.
162
163The --tests= option can be used to adjust the list of tests performed. 161The --tests= option can be used to adjust the list of tests performed.
164 162
165* IPv4, perform the tests with IPv4 addresses (A records) 163* IPv4, perform the tests with IPv4 addresses (A records)
diff --git a/Report-web_0.html b/Report-web_0.html
new file mode 100644
index 0000000..5f56c3f
--- /dev/null
+++ b/Report-web_0.html
@@ -0,0 +1,4 @@
1<!DOCTYPE html>
2<html>
3 <head>
4 <title>apt-panopticon results</title>
diff --git a/Report-web_1.html b/Report-web_1.html
new file mode 100644
index 0000000..a27b076
--- /dev/null
+++ b/Report-web_1.html
@@ -0,0 +1,38 @@
1 <style>
2 .hoverWrapper0:hover #hoverShow0 {
3 display: block;
4 border-style: solid;
5 border-color: fuchsia;
6 }
7 .hoverWrapper0 #hoverShow0 {
8 display: none;
9 background-color: #222222;
10 text-align: left;
11 position: absolute;
12 width: 33%;
13 border-style: solid;
14 border-color: fuchsia;
15 }
16 .hoverWrapper1:hover #hoverShow1 {
17 display: block;
18 border-style: solid;
19 border-color: fuchsia;
20 }
21 .hoverWrapper1 #hoverShow1 {
22 display: none;
23 background-color: #222222;
24 text-align: left;
25 position: absolute;
26 width: 33%;
27 border-style: solid;
28 border-color: fuchsia;
29 }
30 .TTitem {
31 border: 1px solid fuchsia;
32 }
33 </style>
34 </head>
35 <body bgcolor="black" text="white" alink="red" link="aqua" vlink="fuchsia">
36 <h1>Welcome to the apt-panopticon results page.</h1>
37 <p>Here the apt-panopticon system probes into every nook and cranny of the Devuan apt package mirror system, trying to live up to it's name.</p>
38 <p>This is the status of the mirror servers in the Devuan package mirror network.</p>
diff --git a/Report-web_2.html b/Report-web_2.html
new file mode 100644
index 0000000..592e8c6
--- /dev/null
+++ b/Report-web_2.html
@@ -0,0 +1,5 @@
1 <p><font style='background-color:red; color:black'>EXPERIMENTAL CODE - double check all results you see here, and read the logs if it's important.</font></p>
2 <p>The full list of Devuan package mirrors is available at the URL: <a href='https://pkgmaster.devuan.org/mirror_list.txt'>https://pkgmaster.devuan.org/mirror_list.txt</a></p>
3 <p>Due to the nature of the tests, some errors or warnings will be counted several times. &nbsp; The links in the table and DNS list go to the detailed testing logs.</p>
4
5 <hr>
diff --git a/Report-web_3.html b/Report-web_3.html
new file mode 100644
index 0000000..915bd56
--- /dev/null
+++ b/Report-web_3.html
@@ -0,0 +1,11 @@
1 <p><b class="TTitem" title="This is a tooltip with an explanation in it.">Tooltips</b> are things that have an explanation tooltip, hover over them.</p>
2 <p><font color='red'><b>FAILED</b></font> or <font color='lime'><b>OK</b></font> means the tested thing is supported for that mirror.</p>
3 <p><font color='darkred'><b>FAILED*</b></font> or <font color='darkgreen'><b>OK*</b></font> means the tested thing is unsupported for that mirror, but might have been tested anyway.</p>
4 <p><font color='aqua'><b>timeout</b></font> or <font color='blue'><b>timeout</b></font> means the mirror had some timeouts, and tests where not yet aborted. The darker colour means unsupported by the mirror, but tested anyway.</p>
5 <p><font color='aqua'><b>TIMEOUT</b></font> or <font color='blue'><b>TIMEOUT</b></font> means the mirror had too many timeouts, and tests where aborted, so there is no result for this test.</p>
6 <p>NOTE: timeouts may be due to a problem on the testing computer, it might be busy with other things, or be having it's own network problems.</p>
7 <p>NOTE: the speed of the server this checking script is running on might be wildly exaggerated, it doesn't have to go through the Internet to download from itself.</p>
8 <p>The DNS round robin (DNS-RR) column shows the IPs for that mirror, or <font color='grey'><b>no</b></font> if it isn't part of the DNS-RR. &nbsp; The IPs link to the testing log for that IP accessed via the DNS-RR. &nbsp; deb.devuan.org is the DNS-RR itself, so it doesn't get tested directly.</p>
9 <p>The time in the Updated column is how often the mirror updates itself.</p>
10 <p>Mirrors with a <font style='background-color:dimgrey'>grey background</font> are not active (though may be usable as part of the DNS-RR).</p>
11 <p><font color='grey'><b>skip</b></font> means that the test hasn't been written yet.</p>
diff --git a/Report-web_TABLE.html b/Report-web_TABLE.html
new file mode 100644
index 0000000..bad7148
--- /dev/null
+++ b/Report-web_TABLE.html
@@ -0,0 +1,158 @@
1 <table>
2 <tr>
3 <th></th>
4 <th class="TTitem">
5 <div class="hoverWrapper0">
6 FTP
7 <div id="hoverShow0">
8 <h1>FTP tests</h1>
9 <p>The FTP tests have not been written yet.</p>
10 </div>
11 </div>
12 </th>
13 <th class="TTitem">
14 <div class="hoverWrapper0">
15 HTTP
16 <div id="hoverShow0">
17 <h1>HTTP tests</h1>
18 <p>There are two styles of HTTP tests - actual HTTP downloads and HTTP HEAD tests. &nbsp;
19 Actual downloads happen when other tests need the files to be downloaded. &nbsp;
20 HEAD tests are where apt-panopticon probes things in detail. &nbsp;
21 For each mirror (including the DNS round robin domain), and for each IP of that mirror -
22 </p>
23 <ul>
24 <li>Chose a small collection of package files to test, and the Release files for each release.</li>
25 <li>Send a HTTP HEAD request for each of those files.</li>
26 <li>Carefully inspect and log the response.</li>
27 <li>Retry the request if needed, and log that.</li>
28 <li>If the mirror replies with a redirect to the same mirror, then try that and keep checking.</li>
29 <li>If the mirror replies with a redirect to a different server, then probe that mirror the same way.</li>
30 <li>If a redirect loop is detected, log that and give up.</li>
31 </ul>
32 </div>
33 </div>
34 </th>
35 <th class="TTitem">
36 <div class="hoverWrapper0">
37 HTTPS
38 <div id="hoverShow0">
39 <h1>HTTPS tests</h1>
40 <p>The HTTPS tests are very similar to the HTTP tests detailed above, though obviously they are tried with HTTPS requests instead of HTTP requests. &nbsp;
41 The validity of the HTTPS certificate for each server is tested as well.
42 </p>
43 </div>
44 </div>
45 </th>
46 <th class="TTitem">
47 <div class="hoverWrapper0">
48 RSYNC
49 <div id="hoverShow0">
50 <h1>RSYNC tests</h1>
51 <p>The RSYNC tests have not been written yet.</p>
52 </div>
53 </div>
54 </th>
55 <th class="TTitem">
56 <div class="hoverWrapper0">
57 DNS round robin
58 <div id="hoverShow0">
59 <h1>DNS round robin</h1>
60 <p>The "DNS round robin" column lists the IP addresses for each mirror that is part of the DNS round robin, or DNS-RR. &nbsp;
61 The IPs are linked to the log for that specific IP when used via the DNS round robin, and is followed by the number of ERRORs, WARNINGs, or TIMEOUTs if any.
62 </p>
63 </div>
64 </div>
65 </th>
66 <th class="TTitem">
67 <div class="hoverWrapper0">
68 Protocol
69 <div id="hoverShow0">
70 <h1>Protocol tests</h1>
71 <p>The Protocol test will give a warning if the protocol is changed during a redirect, HTTP -> HTTPS for example. &nbsp;
72 While apt HTTPS transport is now the default in Beowulf / Buster, not everyone with an older release will have that installed,
73 so redirecting HTTP to HTTPS will break apt for those people. &nbsp;
74 An ERROR is given instead if that happens for mirrors in the DNS round robin. &nbsp;
75 Servers in the DNS round robin will not have the HTTPS certificate for the round robin domain, so redirecting to HTTPS for that is an ERROR.
76 </p>
77 </div>
78 </div>
79 </th>
80 <th class="TTitem">
81 <div class="hoverWrapper0">
82 Redirects
83 <div id="hoverShow0">
84 <h1>Redirect tests</h1>
85 <p>Mirrors that redirect /DEVUAN/ back out to deb.devuan.org is an ERROR. &nbsp;
86 /DEBIAN-SECURITY/ packages must be redirected to a Debian mirror, specifically a mirror hosting Debian security updates. &nbsp;
87 However, some Devuan mirrors might also be Debian mirrors, so this is just a WARNING.
88 </p>
89 </div>
90 </div>
91 </th>
92 <th class="TTitem">
93 <div class="hoverWrapper0">
94 URL sanity
95 <div id="hoverShow0">
96 <h1>URL sanity tests</h1>
97 <p>The URL sanity test replaces "/" in URLS with "///", to see if the mirror can cope with that. &nbsp;
98 This might happen due to a minor mis-configuration by the apt user, but decent web servers should cope with that. &nbsp;
99 The result for a mirror that does not cope is a failed download for that user, so this is an ERROR.
100 </p>
101 </div>
102 </div>
103 </th>
104 <th class="TTitem">
105 <div class="hoverWrapper0">
106 Integrity
107 <div id="hoverShow0">
108 <h1>Integrity tests</h1>
109 <p>Actually download files, then check things like PGP keys, SHA256 check sums, and file size. &nbsp;
110 For packages, pick the smallest one that has been recently updated.
111 </p>
112 </div>
113 </div>
114 </th>
115 <th class="TTitem" colspan='2'>
116 <div class="hoverWrapper0">
117 Updated
118 <div id="hoverShow0">
119 <h1>Updated tests</h1>
120 <p>Make sure the Release files are up to date by checking their internal "Date" field. &nbsp;
121 If they are up to date, download and check updated Packages.xz files, and actual packages. &nbsp;
122 For actual packages, pick the smallest one that has been recently updated.
123 </p>
124 <p>Also shown is the mirrors scheduled time between updates, with "m" meaning minutes and "h" meaning hours. &nbsp;
125 If this is showing a WARNING, it's likely that the mirror hasn't updated yet, but is due to update after this test was run.
126 </p>
127 </div>
128 </div>
129 </th>
130 <th class="TTitem" colspan='2'>
131 <div class="hoverWrapper0">
132 Speed range
133 <div id="hoverShow0">
134 <h1>Speed range</h1>
135 <p>The speed test tries to guess at a minimum and maximum speed range for each mirror. &nbsp;
136 It does this by measuring the reported speeds from the curl commands that actually download files. &nbsp;
137 Since apt-panopticon is trying hard to download everything from all mirrors all at the same time, this guess will be low. &nbsp;
138 Also, the computer running the apt-panopticon might have a network connection that is busy with other things. &nbsp;
139 Not to mention that obviously if the computer this is running on is one of the mirrors, it might get a wildly exaggerated speed,
140 not having to go through the rest of the Internet tends to help. &nbsp;
141 Finally, the tested mirror may have a bigger network connection than the computer running the test, so wont show it's true maximum. &nbsp;
142 So take this speed measurement with a grain of salt, it's more of an indication, the <a href="../apt-panopticon_cgp/index.php">full graphs</a> might be more useful.
143 </p>
144 </div>
145 </div>
146 </th>
147 <th class="TTitem" colspan='2'>
148 <div class="hoverWrapper0">
149 Weekly statistics
150 <div id="hoverShow0">
151 <h1>Weekly statistics</h1>
152 <p>This is the percentage of time, over the last week, that the mirror was up, and the percentage of time the mirror was up to date. &nbsp;
153 Note that if the mirror has a low uptime, then there wasn't much chance to check if it was up to date.
154 </p>
155 </div>
156 </div>
157 </th>
158 </tr>
diff --git a/apt-panopticon-report-email-web.lua b/apt-panopticon-report-email-web.lua
index e59f543..1cb74fd 100755
--- a/apt-panopticon-report-email-web.lua
+++ b/apt-panopticon-report-email-web.lua
@@ -15,11 +15,6 @@ APT.mirrors = loadfile("results/mirrors.lua")()
15APT.debians = loadfile("results/debians.lua")() 15APT.debians = loadfile("results/debians.lua")()
16 16
17 17
18local lnk = function(name, link)
19 if nil == link then link = name end
20 return name .. " <a href='explanations.html#" .. link .. "'>*</a>"
21end
22
23local revDNS = function(hosts, dom, IP) 18local revDNS = function(hosts, dom, IP)
24 if APT.options.roundRobin.value ~= dom then 19 if APT.options.roundRobin.value ~= dom then
25 if nil ~= hosts[APT.options.roundRobin.value] then 20 if nil ~= hosts[APT.options.roundRobin.value] then
@@ -210,12 +205,18 @@ local DNSrrTest = function(hosts, k)
210 return dns 205 return dns
211end 206end
212 207
208
209local copyHTMLbit = function(web, file)
210 local rfile, e = io.open(file, "r")
211 if nil == rfile then W("opening " .. file .. " file - " .. e) else
212 for line in rfile:lines("*l") do
213 web:write(line .. '\n')
214 end
215 end
216end
217
213local makeTable = function(web, hosts) 218local makeTable = function(web, hosts)
214 web:write("<table>\n<tr><th></th><th>" .. lnk('FTP') .. "</th><th>" .. lnk('HTTP') .. "</th><th>" .. lnk('HTTPS') .. "</th><th>" .. lnk('RSYNC') .. "</th>" .. 219 copyHTMLbit(web, "Report-web_TABLE.html")
215 "<th>" .. lnk('DNS round robin', 'DNS-RR') .. "</th><th>" .. lnk('Protocol') .. "</th><th>" .. lnk('Redirects') ..
216 "</th><th>" .. lnk('URL sanity', 'URL-Sanity') .. "</th><th>" .. lnk('Integrity') .. "</th>" ..
217 "<th colspan='2'>" .. lnk('Updated') .. "</th><th colspan='2'>" .. lnk('Speed range', 'Speed') .. "</th>" ..
218 "<th colspan='2'>" .. lnk('Weekly statistics', 'Weekly') .. "</th></tr>\n")
219 local bg = '' 220 local bg = ''
220 for k, v in APT.orderedPairs(hosts) do 221 for k, v in APT.orderedPairs(hosts) do
221 if '' == bg then bg = " style='background-color:#111111'" else bg = '' end 222 if '' == bg then bg = " style='background-color:#111111'" else bg = '' end
@@ -344,20 +345,21 @@ APT.html = false
344local email, e = io.open("results/Report-email.txt", "w+") 345local email, e = io.open("results/Report-email.txt", "w+")
345if nil == email then C("opening mirrors file - " .. e) else 346if nil == email then C("opening mirrors file - " .. e) else
346 email:write( "Dear Mirror Admins,\n\n" .. 347 email:write( "Dear Mirror Admins,\n\n" ..
347 "This is the status of the mirror servers in the Devuan package mirror network.\n\n" .. 348 "This is a summary of the status of the mirror servers in the \nDevuan package mirror network.\n\n" ..
348 "EXPERIMENTAL CODE - double check all results you see here, and read the logs if it's important." .. 349 "EXPERIMENTAL CODE - double check all results you see here, \nand read the logs if it's important.\n\n" ..
349 "The full list of Devuan package mirrors is available at the URL:\n\n" .. 350 "The full list of Devuan package mirrors is available at the URL:\n\n" ..
350 " https://pkgmaster.devuan.org/mirror_list.txt\n\n" .. 351 " https://pkgmaster.devuan.org/mirror_list.txt\n\n" ..
351 'Please contact "mirrors@devuan.org" if any of the information \nin the file above needs to be amended. \n\n' .. 352 'Please contact "mirrors@devuan.org" if any of the information \nin the file above needs to be amended.\n\n' ..
352 "The full results of the mirror checking is available at the URL:\n\n" .. 353 "The full results of the mirror checking is available at the URLs:\n\n" ..
353 " https://sledjhamr.org/apt-panopticon/results/Report-web.html\n\n" .. 354 " https://borta.devuan.dev/apt-panopticon/results/Report-web.html\n (updated once every hour)\n" ..
355 " https://sledjhamr.org/apt-panopticon/results/Report-web.html\n (updated once every ten minutes)\n\n" ..
354 "Due to the nature of the tests, some errors or warnings will be \ncounted several times. " .. 356 "Due to the nature of the tests, some errors or warnings will be \ncounted several times. " ..
355 "Refer to the logs on the web page for details.\n\n" .. 357 "Refer to the logs on the web page for details.\n\n" ..
356 "Please see below the current status of the Devuan Package Mirror \nnetwork:\n\n" .. 358 "Please see below the current status of the Devuan Package Mirror \nnetwork:\n\n" ..
357 "==== package mirror status " .. os.date("!%F %H:%M") .. " GMT ====\n" .. 359 "==== package mirror status " .. os.date("!%F %H:%M") .. " GMT ====\n" ..
358 "[skip] means that the test hasn't been written yet.\n\n") 360 "[skip] means that the test hasn't been written yet.\n\n")
359 for k, v in APT.orderedPairs(APT.mirrors) do 361 for k, v in APT.orderedPairs(APT.mirrors) do
360 email:write(k .. "....\n") 362 email:write(k .. "..\n")
361 local results = APT.collateAll(APT.mirrors, 'results', k) 363 local results = APT.collateAll(APT.mirrors, 'results', k)
362 local ftp = "[skip]" 364 local ftp = "[skip]"
363 local http = status(APT.mirrors, k, results, "http") 365 local http = status(APT.mirrors, k, results, "http")
@@ -372,13 +374,19 @@ if nil == email then C("opening mirrors file - " .. e) else
372 374
373 -- DNS-RR test. 375 -- DNS-RR test.
374 if (APT.options.roundRobin.value ~= k) and (nil ~= APT.mirrors[APT.options.roundRobin.value]) then 376 if (APT.options.roundRobin.value ~= k) and (nil ~= APT.mirrors[APT.options.roundRobin.value]) then
375 dns = " DNS-RR: " .. dns 377 dns = " DNS-RR: " .. dns .. "\n"
376 end 378 end
377 379
378 email:write( " ftp: " .. ftp .. " http: " .. http .. " https: " .. https .." rsync: " .. rsync .. "\n" .. 380 email:write( " ftp: " .. ftp .. "\n" ..
379 " " .. dns .. "\n" .. 381 " http: " .. http .. "\n" ..
380 " Protocol: " .. protocol .. " URL-sanity: " .. sanity .. " Integrity: " .. integrity .. "\n" .. 382 " https: " .. https .. "\n" ..
381 " Updated: " .. updated .. "\n") 383 " rsync: " .. rsync .. "\n" ..
384 dns ..
385 " Protocol: " .. protocol .. "\n" ..
386 " Redirects: " .. redirects .. "\n" ..
387 " URL-sanity: " .. sanity .. "\n" ..
388 " Integrity: " .. integrity .. "\n" ..
389 " Updated: " .. updated .. "\n")
382 end 390 end
383 email:write( "\n==== faulty mirrors: ====\n" .. faulty) 391 email:write( "\n==== faulty mirrors: ====\n" .. faulty)
384 email:write( "\n-------------------------\n\n" .. 392 email:write( "\n-------------------------\n\n" ..
@@ -387,7 +395,7 @@ if nil == email then C("opening mirrors file - " .. e) else
387 "You can get the source code from https://sledjhamr.org/cgit/apt-panopticon/about/ (main repo)\n" .. 395 "You can get the source code from https://sledjhamr.org/cgit/apt-panopticon/about/ (main repo)\n" ..
388 "and from https://git.devuan.org/onefang/apt-panopticon' (Devuan repo).\n" .. 396 "and from https://git.devuan.org/onefang/apt-panopticon' (Devuan repo).\n" ..
389 "You can get the cgp graphing source code from https://sledjhamr.org/cgit/apt-panopticon_cgp/about/ (main repo)\n" .. 397 "You can get the cgp graphing source code from https://sledjhamr.org/cgit/apt-panopticon_cgp/about/ (main repo)\n" ..
390 "and https://git.devuan.org/onefang/apt-panopticon_cgp (Devuan repo)\n" .. 398 "and https://git.devuan.org/onefang/apt-panopticon_cgp (Devuan repo)\n\n" ..
391 "Love\n\n" .. 399 "Love\n\n" ..
392 "The Dev1Devs\n\n") 400 "The Dev1Devs\n\n")
393 email:close() 401 email:close()
@@ -454,60 +462,33 @@ APT.rrd.graph('results/speed.png', '--start', 'now-2w', '--end', 'now', '-t', 'S
454unpack(g)) 462unpack(g))
455 463
456 464
457
458results = {} 465results = {}
459m = {} 466m = {}
460faulty = "" 467faulty = ""
461APT.html = true 468APT.html = true
462local web, e = io.open("results/Report-web.html", "w+") 469local web, e = io.open("results/Report-web.html", "w+")
463if nil == web then C("opening mirrors file - " .. e) else 470if nil == web then C("opening mirrors file - " .. e) else
464 web:write( "<html><head><title>apt-panopticon results</title>\n") 471 copyHTMLbit(web, "Report-web_0.html")
465 if 0 < tonumber(APT.options.refresh.value) then 472 if 0 < tonumber(APT.options.refresh.value) then
466 web:write('<meta http-equiv="refresh" content="' .. APT.options.refresh.value .. '">\n') 473 web:write('<meta http-equiv="refresh" content="' .. APT.options.refresh.value .. '">\n')
467 end 474 end
468 web:write( '</head><body bgcolor="black" text="white" alink="red" link="aqua" vlink="fuchsia">' .. 475 copyHTMLbit(web, "Report-web_1.html")
469 "<h1>Welcome to the apt-panopticon results page.</h1>\n" ..
470 "<p>Here the apt-panopticon system probes into every nook and cranny of the Devuan apt package mirror system, trying to live up to it's name.</p>" ..
471 "<p>This is the status of the mirror servers in the Devuan package mirror network.</p>\n")
472 if 0 < tonumber(APT.options.refresh.value) then 476 if 0 < tonumber(APT.options.refresh.value) then
473 web:write( '<p>This page will refresh every ' .. (APT.options.refresh.value / 60) .. ' minutes.</p>') 477 web:write( '<p>This page will refresh every ' .. (APT.options.refresh.value / 60) .. ' minutes.</p>')
474 end 478 end
475 web:write( "<p><font style='background-color:red; color:black'>EXPERIMENTAL CODE - double check all results you see here, and read the logs if it's important.</font></p>" .. 479 copyHTMLbit(web, "Report-web_2.html")
476 "<p>The full list of Devuan package mirrors is available at the URL: " .. 480 web:write("\n<h2>==== package mirror status " .. os.date("!%F %H:%M") .. " GMT ====</h2>\n")
477 "<a href='https://pkgmaster.devuan.org/mirror_list.txt'>https://pkgmaster.devuan.org/mirror_list.txt</a></p>\n" .. 481 copyHTMLbit(web, "Report-web_3.html")
478 "<p>Due to the nature of the tests, some errors or warnings will be counted several times. &nbsp; " ..
479 "The links in the table and DNS list go to the detailed testing logs.</p>\n\n" ..
480 "<hr>\n<h2>==== package mirror status " .. os.date("!%F %H:%M") .. " GMT ====</h2>\n" ..
481
482 "<p><font color='red'><b>FAILED</b></font> or <font color='lime'><b>OK</b></font>" ..
483 " means the tested thing is supported for that mirror.</p>\n" ..
484 "<p><font color='darkred'><b>FAILED*</b></font> or <font color='darkgreen'><b>OK*</b></font>" ..
485 " means the tested thing is unsupported for that mirror, but might have been tested anyway.</p>\n" ..
486 "<p><font color='aqua'><b>timeout</b></font> or <font color='blue'><b>timeout</b></font>" ..
487 " means the mirror had some timeouts, and tests where not yet aborted. The darker colour means unsupported by the mirror, but tested anyway.</p>" ..
488 "<p><font color='aqua'><b>TIMEOUT</b></font> or <font color='blue'><b>TIMEOUT</b></font>" ..
489 " means the mirror had too many timeouts, and tests where aborted, so there is no result for this test.</p>" ..
490 "<p>NOTE: timeouts may be due to a problem on the testing computer, it might be busy with other things, or be having it's own network problems..</p>" ..
491 "<p>NOTE: the speed of the server this checking script is running on might be wildly exaggerated, it doesn't have to go through the Internet to download from itself.</p>" ..
492 "<p>The DNS round robin (DNS-RR) column shows the IPs for that mirror, or <font color='grey'><b>no</b></font> if it isn't part of the DNS-RR. &nbsp; " ..
493 "The IPs link to the testing log for that IP accessed via the DNS-RR. &nbsp; " ..
494 APT.options.roundRobin.value .. " is the DNS-RR itself, so it doesn't get tested directly.</p>\n" ..
495 "<p>The time in the Updated column is how often the mirror updates itself.</p>" ..
496 "<p>Mirrors with a <font style='background-color:dimgrey'>grey background</font> are not active (though may be usable as part of the DNS-RR).</p>\n" ..
497 "<p><font color='grey'><b>skip</b></font> means that the test hasn't been written yet.</p>\n"
498 )
499
500 makeTable(web, APT.mirrors) 482 makeTable(web, APT.mirrors)
501 web:write( "<h2>==== faulty mirrors: ====</h2>\n" .. faulty) 483 web:write( "<h2>==== faulty mirrors: ====</h2>\n" .. faulty)
502 web:write( "<br>\n<br>\n<h2>==== DNS, graphs <a href='explanations.html#Graphs'>*</a>, and logs: ====</h2>\n") 484 web:write( "<br>\n<br>\n<h2>==== DNS, links to graphs, and links to logs: ====</h2>\n")
503
504 m = makeIPlist(APT.mirrors) 485 m = makeIPlist(APT.mirrors)
505 web:write( "<p>This lists each mirror, and the DNS entries for that mirror. &nbsp; " .. 486 web:write( "<p>This lists each mirror, and the DNS entries for that mirror. &nbsp; " ..
506 "The links point to the testing log files (" .. logCount("apt-panopticon") .. ") for each domain name / IP combination that was tested. &nbsp; " .. 487 "The IP links point to the testing log files (the overall log is " .. logCount("apt-panopticon") .. ") for each domain name / IP combination that was tested. &nbsp; " ..
507 "If a mirror has a CNAME, that CNAME is listed along with that CNAMEs DNS entries. &nbsp; " .. 488 "If a mirror has a CNAME, that CNAME is listed along with that CNAMEs DNS entries. &nbsp; " ..
508 APT.options.roundRobin.value .. " is the DNS round robin, which points to the mirrors that are part of the DNS-RR. &nbsp; " .. 489 APT.options.roundRobin.value .. " is the DNS round robin, which points to the mirrors that are part of the DNS-RR. &nbsp; " ..
509 "If an IP is part of the DNS-RR, it is marked with '<font color='purple'><b>DNS-RR</b></font>' &nbsp; " .. 490 "If an IP is part of the DNS-RR, it is marked with '<font color='purple'><b>DNS-RR</b></font>' &nbsp; " ..
510 APT.options.referenceSite.value .. " is the master mirror, all the others sync to it. &nbsp; " .. 491 APT.options.referenceSite.value .. " is the master mirror, all the others copy files from it. &nbsp; " ..
511 "</p>\n" 492 "</p>\n"
512 ) 493 )
513 web:write(APT.dumpTableHTML(m, "")) 494 web:write(APT.dumpTableHTML(m, ""))
@@ -524,7 +505,7 @@ if nil == web then C("opening mirrors file - " .. e) else
524 ) 505 )
525 506
526 makeTable(web, APT.debians) 507 makeTable(web, APT.debians)
527 web:write( "<br>\n<br>\n<h2>==== Debian DNS, graphs <a href='explanations.html#Graphs'>*</a>, and logs: ====</h2>\n") 508 web:write( "<br>\n<br>\n<h2>==== Debian DNS, links to graphs, and links to logs: ====</h2>\n")
528 509
529 m = makeIPlist(APT.debians) 510 m = makeIPlist(APT.debians)
530 web:write(APT.dumpTableHTML(m, "")) 511 web:write(APT.dumpTableHTML(m, ""))
diff --git a/apt-panopticon.lua b/apt-panopticon.lua
index a36b229..60e73f4 100755
--- a/apt-panopticon.lua
+++ b/apt-panopticon.lua
@@ -1129,7 +1129,6 @@ else
1129 os.execute("rm -f results/*.html 2>/dev/null") 1129 os.execute("rm -f results/*.html 2>/dev/null")
1130 os.execute("rm -f results/*.txt 2>/dev/null") 1130 os.execute("rm -f results/*.txt 2>/dev/null")
1131 end 1131 end
1132 os.execute('ln -s ../explanations.html results/explanations.html 2>/dev/null')
1133 1132
1134 if not APT.logOpen('apt-panopticon') then return end 1133 if not APT.logOpen('apt-panopticon') then return end
1135 I("Starting tests " .. table.concat(APT.options.tests.value, ", ")) 1134 I("Starting tests " .. table.concat(APT.options.tests.value, ", "))
diff --git a/explanations.html b/explanations.html
deleted file mode 100644
index 73820c8..0000000
--- a/explanations.html
+++ /dev/null
@@ -1,96 +0,0 @@
1<html><head><title>apt-panopticon explanations</title>
2</head><body bgcolor="black" text="white"><h1>Welcome to the apt-panopticon explanations page.</h1>
3<p>The purpose of this page is to try to explain everything shown on <a href="Report-web.html">the results page</a>,
4so that the results page can link to the various parts for the curious.<p>
5
6<hr><h1 id="IPv4">IPV4 tests</h1>
7<p>Tests are tested over IPv4 if the mirror has IPv4 addresses.</p>
8
9<hr><h1 id="IPv6">IPv6</h1>
10<p>Tests are tested over IPv6 if the mirror has IPv6 addresses.</p>
11
12<hr><h1 id="FTP">FTP tests</h1>
13<p>The FTP tests have not been written yet.</p>
14
15<hr><h1 id="HTTP">HTTP tests</h1>
16<p>There are two styles of HTTP tests - actual HTTP downloads and HTTP HEAD tests. &nbsp;
17Actual downloads happen when other tests need the files to be downloaded. &nbsp;
18HEAD tests are where apt-panopticon probes things in detail. &nbsp;
19For each mirror (including the DNS round robin domain), and for each IP of that mirror -
20</p>
21<ul>
22<li>Chose a small collection of package files to test, and the Release files for each release.</li>
23<li>Send a HTTP HEAD request for each of those files.</li>
24<li>Carefully inspect and log the response.</li>
25<li>Retry the request if needed, and log that.</li>
26<li>If a redirect loop is detected, log that and give up.</li>
27<li>If the mirror replies with a redirect to the same mirror, then try that and keep checking.</li>
28<li>If the mirror replies with a redirect to a different server, then probe that mirror the same way.</li>
29</ul>
30
31<hr><h1 id="HTTPS">HTTPS tests</h1>
32<p>The HTTPS tests are very similar to the HTTP tests detailed above, though obviously they are tried with HTTPS requests intead of HTTP requests. &nbsp;
33The validity of the HTTPS certificate for each server is tested as well.
34</p>
35
36<hr><h1 id="RSYNC">rsync tests</h1>
37<p>The RSYNC tests have not been written yet.</p>
38
39<hr><h1 id="DNS-RR">DNS round robin tests</h1>
40<p>The "DNS round robin" column lists the IP addresses for each mirror that is part of the DNS round robin, or DNS-RR. &nbsp;
41The IPs are linked to the log for that specific IP when used via the DNS round robin, and is followed by the number of errors, warnings, or timeouts of any.
42</p>
43
44<hr><h1 id="Redirects">Redirect tests</h1>
45<p>Mirrors that redirect /DEVUAN/ back out to deb.devuan.org is an ERROR. &nbsp;
46/DEBIAN-SECURITY/ packages must be redirected to a Debian mirror, specifically a mirror hosting Debian security updates. &nbsp;
47However, some Devuan mirrors might also be Debian mirrors, so this is just a WARNING.
48</p>
49
50<hr><h1 id="Protocol">Protocol tests</h1>
51<p>The Protocol test will give a warning if the protocol is changed during a redirect, HTTP -> HTTPS for example. &nbsp;
52While apt HTTPS transport is now the default in Beowulf / Buster, not everyone with an older release will have that installed,
53so redirecting HTTP to HTTPS will break apt for those people. &nbsp;
54An error is given instead if that happens for mirrors in the DNS round robin. &nbsp;
55Servers in the DNS round robin will not have the HTTPS certificate for the round robin domain, so redirecting to HTTPS for that is a mistake.
56</p>
57
58<hr><h1 id="URL-Sanity">URL sanity tests</h1>
59<p>The URL sanity test replaces "/" in URLS with "///", to see if the mirror can cope with that. &nbsp;
60This might happen due to a minor mis-configuration by the apt user, but decent web servers should cope with that. &nbsp;
61The result for a mirror that does not cope is a failed download for that use, so this is an error.
62</p>
63
64<hr><h1 id="Integrity">Integrity tests</h1>
65<p>Actually download files, then check things like PGP keys, SHA256 check sums, and file size. &nbsp;
66For actual packages, pick the smallest one that has been recently updated.
67</p>
68
69<hr><h1 id="Updated">Updated tests</h1>
70<p>Make sure the Release files are up to date by checking their internal "Date" field. &nbsp;
71If they are up to date, download and check updated Packages.xz files, and actual packages. &nbsp;
72For actual packages, pick the smallest one that has been recently updated.
73</p>
74<p>Also shown is the mirrors scheduled time between updates, with "m" meaning minutes and "h" meaning hours.
75</p>
76
77<hr><h1 id="Speed">Speed test</h1>
78<p>The speed test tries to guess at a minimum and maximum speed range for each mirror. &nbsp;
79It does this by measuring the reported speeds from the curl commands that actually download files. &nbsp;
80Since apt-panopticon is trying hard to download everything from all mirrors all at the same time, this guess will be low. &nbsp;
81Also, the computer running the apt-panopticon might have a network connection that is busy with other things. &nbsp;
82Not to mention that obviously if the computer this is running on is one of the mirrors, it might get a wildly exaggerated speed,
83not haaving to go through the rest of the Internet tends to help. &nbsp;
84Finally, the tested mirror may have a bigger network connection than the computer running the test, so wont show it's true maximum. &nbsp;
85So take this speed measurement with a grain of salt, it's more of an indication, the <a href="../apt-panopticon_cgp/index.php">full graphs</a> might be useful.
86</p>
87
88<hr><h1 id="Weekly">Weekly averages</h1>
89<p>This is the percentage of time the mirror was up, and the percentage of time the mirror was up to date. &nbsp;
90Note that if the mirror has a low uptime, then there wasn't much chance to check if it was up to date.
91</p>
92
93<hr><h1 id="Graphs">Graphs</h1>
94<p>A link to the graphs for this mirror.</p>
95
96</body></html>
generated by cgit v1.1 at 2024-05-14 23:35:37 +0000