/
maintenance.txt
131 lines (98 loc) · 5.4 KB
/
maintenance.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
aurweb Maintenance
==================
Components
----------
aurweb has three user-facing components.
The Git/SSH interface can be used to update package repositories and for basic
package maintenance from the command-line. More details can be found in
`doc/git-interface.txt`.
The web interface can be used to browse packages, view package details, manage
aurweb accounts, add comments, vote for packages, flag packages, and submit
requests. Package Maintainers can update package maintainers and delete/merge
packages. The web interface also includes an area for Package Maintainers to post
AUR-related proposals and vote on them.
The RPC interface can be used to query package information via HTTP.
Installation
------------
The web backend requires a web server and an SQL database. The Git/SSH
interface requires Python, several Python modules and an up-to-date version of
Git. APCu or memcached can be used to reduce load on the database server.
All dependencies and the full installation process are described in `INSTALL`.
Updates
-------
The `enable-maintenance` option (in the configuration file, usually located at
`/etc/aurweb/config`) can be used to switch aurweb into maintenance mode. This
disables both the Git/SSH interface and the web interface. The
`maintenance-exceptions` variable can be used to reactivate access for certain
IP addresses. Since changes to the database schema might temporarily break
parts of the backend, it is recommended to always enable maintenance mode
before performing an upgrade.
To simplify the upgrade process, changes in the database schema (and other
changes that require manual interaction) are documented in `upgrading/`. An
exception are additions to the configuration file. It is recommended to always
compare `/etc/aurweb/config` to `conf/config.defaults` when upgrading to a new
release.
Moreover, the aurweb Python modules and translations need to be reinstalled
with every upgrade. To this end, run `python3 setup.py install` from the aurweb
source tree and run `make install` in the `po/` subdirectory.
Don't forget to always test all basic features first, then disable maintenance
mode after performing an upgrade.
Maintenance Scripts
-------------------
aurweb includes scheduled maintenance routines to perform expensive
computations and clean up the database:
* aurweb-aurblup parses binary repositories and updates the `OfficialProviders`
table. This table is used to identify AUR packages that depend on packages in
the official repositories. It is also used to prevent users from uploading
packages that are in the official repositories already.
* aurweb-votereminder sends out reminders if the voting period for a
Package Maintainer proposal ends soon.
* aurweb-popupdate is used to recompute the popularity score of packages.
* aurweb-pkgmaint automatically removes empty repositories that were created
within the last 24 hours but never populated.
* [Deprecated] aurweb-mkpkglists generates the package list files; it takes
an optional --extended flag, which additionally produces multiinfo metadata.
It also generates {archive.gz}.sha256 files that should be located within
mkpkglists.archivedir which contain a SHA-256 hash of their matching
.gz counterpart.
* aurweb-usermaint removes the last login IP address of all users that did not
login within the past seven days.
* aurweb-git-archive generates Git repository archives based on a --spec.
This script is a new generation of aurweb-mkpkglists, which creates and
maintains Git repository versions of the archives produced by
aurweb-mkpkglists. See doc/git-archive.md for detailed documentation.
These scripts can be installed by running `poetry install` and are
usually scheduled using Cron. The current setup is:
----
# Run aurweb-git-archive --spec metadata directly after
# aurweb-mkpkglists so that they are executed sequentially, since
# both scripts are quite heavy. `aurweb-mkpkglists` should be removed
# from here once its deprecation period has ended.
*/5 * * * * poetry run aurweb-mkpkglists [--extended] && poetry run aurweb-git-archive --spec metadata
# Usernames
*/5 * * * * poetry run aurweb-git-archive --spec users
# Package base names
*/5 * * * * poetry run aurweb-git-archive --spec pkgbases
# Package names
*/5 * * * * poetry run aurweb-git-archive --spec pkgnames
1 */2 * * * poetry run aurweb-popupdate
2 */2 * * * poetry run aurweb-aurblup
3 */2 * * * poetry run aurweb-pkgmaint
4 */2 * * * poetry run aurweb-usermaint
5 */12 * * * poetry run aurweb-votereminder
----
Advanced Administrative Features
--------------------------------
Package Maintainers can set the AUR_OVERWRITE environment variable to enable
non-fast-forward pushes to the Git repositories. This feature is documented in
`doc/git-interface.txt`.
Rate limiting is used to prevent users from hammering the RPC interface. The
`request_limit` and `window_length` options in the `ratelimit` section of the
configuration file can be used to configure this feature. Recent accesses are
stored in the `ApiRateLimit` table in the database. See commit 27654af (Add
rate limit support to API, 2018-02-01) for details.
The database contains a `PackageBlacklist` table. Package names added to this
table will be rejected by the SSH/Git interface. This table can only be edited
by a database administrator.
The `Bans` table can be used to ban certain IP addresses from both the web and
Git/SSH interface. This table can only be accessed by a database administrator.