aboutsummaryrefslogtreecommitdiffstatshomepage
diff options
context:
space:
mode:
-rw-r--r--README.md132
1 files changed, 26 insertions, 106 deletions
diff --git a/README.md b/README.md
index 9ff79f2..94eebbf 100644
--- a/README.md
+++ b/README.md
@@ -23,7 +23,15 @@ badgered him about his Python scripts. It should also be much easier to
23use, the previous scripts needed some work before you could run them, 23use, the previous scripts needed some work before you could run them,
24this one you just download and run. 24this one you just download and run.
25 25
26The source code is at [https://sledjhamr.org/cgit/apt-panopticon/](https://sledjhamr.org/cgit/apt-panopticon/) 26After a discussion, it is likely that apt-panopticon will be used to
27automatically decide which Devuan mirrors are in the DNS round robin.
28The details of that are yet to be determined.
29
30
31You can get the source code from [https://sledjhamr.org/cgit/apt-panopticon/](https://sledjhamr.org/cgit/apt-panopticon/) (main repo)
32and [https://git.devuan.org/onefang/apt-panopticon](https://git.devuan.org/onefang/apt-panopticon) (Devuan repo).
33You can get the cgp graphing source code from [https://sledjhamr.org/cgit/apt-panopticon_cgp/]() (main repo)
34and [https://git.devuan.org/onefang/apt-panopticon_cgp](https://git.devuan.org/onefang/apt-panopticon_cgp) (Devuan repo).
27 35
28The issue tracker is at [https://sledjhamr.org/mantisbt/project_page.php?project_id=13](https://sledjhamr.org/mantisbt/project_page.php?project_id=13) 36The issue tracker is at [https://sledjhamr.org/mantisbt/project_page.php?project_id=13](https://sledjhamr.org/mantisbt/project_page.php?project_id=13)
29 37
@@ -56,13 +64,14 @@ If you want to have lots of graphs, also install
56For the apt-panopticon_cgp package, which is used to show the detailed 64For the apt-panopticon_cgp package, which is used to show the detailed
57graphs, you'll need a web server that supports PHP. apt-panopticon_cgp 65graphs, you'll need a web server that supports PHP. apt-panopticon_cgp
58includes some support files for running PHP via CGI, which more web 66includes some support files for running PHP via CGI, which more web
59servers support. You'll need php-cgi for that. 67servers support. You'll need php-cgi for that. It's been tested with
68recent versions of Apache 2 and Lightttpd.
60 69
61 70
62Web installation. 71Web installation.
63----------------- 72-----------------
64 73
65This is a suggestion for installation on a Devuan based web server. 74This is a suggestion for installation on a Devuan based web server.
66 75
67Create - 76Create -
68 77
@@ -74,7 +83,7 @@ Install apt-panopticon and apt-panopticon_cgp there, so you end up with -
74/var/www/html/apt-panopticon/apt-panopticon_cgp 83/var/www/html/apt-panopticon/apt-panopticon_cgp
75 84
76The script update_apt-panopticon is an example script for updating 85The script update_apt-panopticon is an example script for updating
77everything, including commented out commands to update the source code. 86everything, including commented out commands to update the source code.
78The file apt-panopticron is an example crontab file for updating 87The file apt-panopticron is an example crontab file for updating
79everything once every ten minutes. They assume your web server user is 88everything once every ten minutes. They assume your web server user is
80www-data with a group of www-data, and you have a mirror user called 89www-data with a group of www-data, and you have a mirror user called
@@ -82,9 +91,8 @@ mirrors. For mirror operators, that mirrors user would be the owner of
82the mirror files. You can change these to suite yourself. 91the mirror files. You can change these to suite yourself.
83 92
84Once everything is updated, 93Once everything is updated,
85/var/www/html/apt-panopticon/results/Report-web.html 94/var/www/html/apt-panopticon/results/Report-web.html will point to the
86will point to the main web page, and there will be a link at the bottom of 95main web page, and there will be links that point to the detailed graphs.
87that pointing to the detailed graphs.
88 96
89Note that two runs of apt-panopticon have to happen ten minutes apart at 97Note that two runs of apt-panopticon have to happen ten minutes apart at
90least in order to see any data on the graphs. 98least in order to see any data on the graphs.
@@ -98,9 +106,7 @@ Using it.
98 106
99These examples assume you are running it from the source code directory. 107These examples assume you are running it from the source code directory.
100A directory will be created called `results`, it'll be full of log files 108A directory will be created called `results`, it'll be full of log files
101and any files that get downloaded. There will also be `results/email` 109and any files that get downloaded.
102and `results/web` directories, with the notification emails and web pages
103(once I write that bit).
104 110
105Note that unlike typical commands, you can't run single character options 111Note that unlike typical commands, you can't run single character options
106together, so this is wrong - 112together, so this is wrong -
@@ -115,12 +121,12 @@ Just run the script to do all of the tests -
115 121
116 $ ./apt-panopticon.lua 122 $ ./apt-panopticon.lua
117 123
118Which will print any errors. If you don't want to see errors - 124Which will print any CRITICAL errors. If you don't want to see errors -
119 125
120 $ ./apt-panopticon.lua -q 126 $ ./apt-panopticon.lua -q
121 127
122If you want to see warnings as well (as usual, the more `-v` options, the more 128If you want to see other errors, warnings, and even more details as well
123details) - 129(as usual, the more `-v` options, the more details) -
124 130
125 $ ./apt-panopticon.lua -v 131 $ ./apt-panopticon.lua -v
126 132
@@ -184,7 +190,8 @@ and check all of it's IPs, which are the DNS RR mirrors anyway.
184 190
185The mirror_list.txt file also used to select which protocols to test for 191The mirror_list.txt file also used to select which protocols to test for
186each mirror, it will only test those protocols the mirror lists as 192each mirror, it will only test those protocols the mirror lists as
187supporting. 193supporting. Actually it'll test the ones not supported as well, but mark
194them differently in the results.
188 195
189 196
190Options. 197Options.
@@ -200,13 +207,14 @@ Print the version.
200 207
201-v 208-v
202 209
203Print more verbose output. Normally only CRITICAL and ERROR message sare 210Print more verbose output. Normally only CRITICAL messages are printed.
204printed. -v will print WARNING messages as well, -v -v INFO messages, 211-v will print ERROR messages as well, -v -v WARNING messages, -v -v -v
205and -v -v -v DEBUG messages. All messages are logged regardless. 212INFO messages, and -v -v -v -v DEBUG messages. All messages are logged
213regardless.
206 214
207-q 215-q
208 216
209Only print CRITICAL messages. 217Print no output messages.
210 218
211-k 219-k
212 220
@@ -257,91 +265,3 @@ negative argument deselects a report.
257 265
258Select which tests to run. The arguments are comma separated. A 266Select which tests to run. The arguments are comma separated. A
259negative argument deselects a test. Examples are given above. 267negative argument deselects a test. Examples are given above.
260
261
262Theory of operation.
263--------------------
264
265Typically you would call it without any specific mirror mentioned on the
266command line. I'll start the discussion from there.
267
268Create the results directory.
269
270If -k is not given, delete results/*.log.
271
272Delete results/*.check.
273
274touch results/stamp
275
276Open results/apt-panopticon.log for message logging.
277
278Download mirror_list.txt from the reference site. Build a table of
279Active mirrors keyed by the FDQN, include the listed Protocols as a sub
280table. Add the round robin domain name. Resolve all the IPs and add
281them to this table. Write this table to results/mirrors.lua so that the
282forked tests can read it.
283
284checkHost() the reference site first.
285
286Loop through the mirrors table, and checkHost() each one, skipping the
287reference site.
288
289Wait for all forked tests to finish.
290
291Delete results/*.check.
292
293
294The checkHost() function does this -
295
296If there is no second argument, then the host is set to the first
297argument, otherwise the host is the second argument.
298
299Gather the IPs for the host name with the following command -
300
301dig +keepopen +noall +nottlid +answer example.com A example.com AAAA
302example.com CNAME example.com SRV | sort -r | uniq
303
304So it should end up with all the IPV4, IPV6, CNAME, and SRV records for
305that host.
306
307For each IPv4 and IPv6 address, fork a copy of the script something like
308this (including any arguments originally provided to the script) -
309
310ionice -c3 ./apt-panopticon.lua example.com/path x.x.x.x &
311
312ionice -c3 ./apt-panopticon.lua example.com/path [x:x:x:x:x:x] &
313
314For each CNAME, it checkHost() the host, but with the CNAME as a second
315argument.
316
317SRV reconds don't do anything yet, coz I have yet to see one from my test
318environment, so can't test it.
319
320
321Each forked call of the script from above does this -
322
323Open results/example.com_x.x.x.x.log for message logging.
324
325Loads the mirrors table from results/mirrors.lua.
326
327If performing the Integrity or Updated testes, delete results/example.com
328directory, downloads the reference files using wget. While it should
329actually perform the Integrity and Updated tests now, those haven't been
330written yet. Note that currently this downloads 4GB per mirror.
331
332Calls checkHost() with the host as first and second arguments, and
333includes the IP this time. The inclusion of the IP causes checkHost() to
334call checkFiles().
335
336
337checkFiles() will call checkHEAD() for each of the reference files.
338
339
340checkHEAD() uses LuaSocket (or LuaSec for HTTPS) to send a HEAD request
341to the IP, with a Host header set to the original host name. Redirects
342will not be followed by that request. If the request returns a redirect,
343then checkHEAD() is called recursively. If the redirect is to some host
344we are not already checking, we call checkHost() on it, with an IP of
345"redir". This causes checkHost() to bypass the test that would otherwise
346call checkFiles(), instead gathering the IPs and fork as usual.
347