danderson/channel-scripts
0
0
Fork 0
forked from the-distro/channel-scripts
Code Pull requests Activity
132 commits 2 branches 0 tags 295 KiB
Commit graph

132 commits

This branch
This branch
All branches
Author SHA1 Message Date
Eelco Dolstra
b6c049019a
mirror-nixos-branch.pl: Call index-debuginfo 2017-07-07 17:12:25 +02:00
Eelco Dolstra
fcd6250fd0
mirror-channel.pl: Remove 
This is no longer used, since Hydra uploads directly to S3 now. Also,
"nix copy" can copy to S3 now.
 2017-07-07 17:06:21 +02:00
Eelco Dolstra
a5e125330c
generate-programs-index: Check whether symlink targets exist 
Fixes https://github.com/NixOS/nixpkgs/issues/26661
 2017-07-07 16:45:45 +02:00
Eelco Dolstra
74c917454c
Add a command for indexing debug info files by build ID 
For every file "lib/debug/.build-id/<build-id>.debug" in every store
path ending in -debug in a release, this command creates an S3 key
"debuginfo/<build-id>" specifying the location of the NAR that
contains the debug info file for that build ID. Thus,
https://cache.nixos.org/debuginfo allows debug info files to be looked
up by build ID. This allows debug info files to be fetched
automatically by dwarffs [1].

One minor issue is that in theory, multiple store paths could provide
debug info files for a particular build ID. So then when we garbage
collect a store path from the binary cache, we migth invalidate a
debug info link even though there are non-GC'ed store paths that
provide the file. Ah well.

I could have used HTTP redirects, but they're kind of a pain to
configure with S3.

[1] https://github.com/edolstra/dwarffs
 2017-07-07 15:33:27 +02:00
Eelco Dolstra
a91fb566a9
Factor out the file cache 2017-07-06 14:53:52 +02:00
Eelco Dolstra
3daca6b592
Update generate-programs-index to handle .ls 2017-04-18 16:41:11 +02:00
Eelco Dolstra
dd0d2d2ca6
Update build 2017-04-18 15:26:38 +02:00
Eelco Dolstra
a5368b6239 Merge pull request #12 from primeos/master 
Fix the displayed hash name (SHA-512 -> SHA-256)
 2017-03-31 12:04:05 +02:00
Michael Weiss
c3e8e3badb Fix the displayed hash name (SHA-512 -> SHA-256) 
The HTML pages for each channel are showing the SHA-256 hash of each
file but the table header actually referred to it as an "SHA-512 hash".
 2017-03-31 06:39:24 +02:00
Eelco Dolstra
1c07d6dd08
Lock the Git repo 2016-12-05 11:50:17 +01:00
Eelco Dolstra
2fd4d6beea Vacuum programs.sqlite 
This cuts a few hundred KiB from the database.
 2016-11-10 18:27:04 +01:00
Eelco Dolstra
6883275653 Ignore package listings with malformed UTF-8 
For example, https://cache.nixos.org/5q3s4vfwwd9n9499yg11ydziwxgn644m.ls.xz
contains the string "bokm<E5>l.alias". This causes nlohmann::json to
barf.
 2016-11-10 18:10:08 +01:00
Eelco Dolstra
2176c939fc Compute FS closures in parallel 
This is much faster.
 2016-11-10 18:10:08 +01:00
Eelco Dolstra
b74e03a870 Revert "Disable programs.sqlite generation" 
This reverts commit 7f2bbc884c.
 2016-11-10 18:10:08 +01:00
Eelco Dolstra
ba19471e5b Use the .ls.xz listings generated by Hydra to speed up building programs.sqlite 
This reduces the time to build the index for the regular (non-small) channel from ~10 hours to ~4 minutes.

Fixes #9.
 2016-11-10 18:10:03 +01:00
Eelco Dolstra
6391c4dcec Fix check for release existance 
We no longer have a github-link file, so this check always failed. As
a result, we were constantly re-uploading releases.
 2016-10-21 20:10:53 +02:00
Eelco Dolstra
038c8ee8d4 Update the channels repo early 
This way, users of the nixpkgs-channels repo don't have to wait for
all the slow stuff to finish (like index generation and copying of
ISOs).
 2016-10-21 19:22:05 +02:00
Eelco Dolstra
7f2bbc884c Disable programs.sqlite generation 2016-10-21 18:27:28 +02:00
Eelco Dolstra
eea0729481 Generate a simple directory listing 2016-09-29 11:50:14 +02:00
Eelco Dolstra
6cb926311e Only update the release link if the release has changed 2016-09-29 10:55:11 +02:00
Eelco Dolstra
fcf5a77736 Add an empty file at the release prefix for nix-channel compatibility 
nix-channel expects the prefix to exist (not return a 404).
 2016-09-26 19:50:36 +02:00
Eelco Dolstra
d3d33d5b25 Store releases in S3 
Fixes #7.
 2016-09-26 18:06:51 +02:00
Eelco Dolstra
becf1f2ae3 Atomically update channels/.htaccess 
Otherwise we might clobber the redirects if the disk is full, and in
any case there is a time window in which .htaccess is in an
inconsistent state.

Fixes https://github.com/NixOS/nixpkgs/issues/18544.
 2016-09-12 23:20:12 +02:00
Eelco Dolstra
8a0ed20e6d Improve error message 2016-08-22 17:26:27 +02:00
Eelco Dolstra
76ca1da3ee Add $out/bin to $PATH 2016-08-22 17:18:44 +02:00
Eelco Dolstra
c981c37438 Add README 2016-08-22 17:18:29 +02:00
Eelco Dolstra
ab086b22ef Revive programs.sqlite generation 
Fixes #4.
 2016-08-11 14:41:26 +02:00
Eelco Dolstra
8f7fbdaff7 Die tabs die 2016-08-08 17:32:24 +02:00
Eelco Dolstra
72501d545d Remove obsolete binary patch generation scripts 2016-04-13 11:42:52 +02:00
Eelco Dolstra
852e1b6bf2 Drop 32-bit images (except iso-minimal) 2016-03-17 12:21:13 +01:00
Eelco Dolstra
000a9f2766 Workaround for https://github.com/NixOS/nix/issues/792 2016-03-07 11:12:24 +01:00
Eelco Dolstra
373d8b7b38 Fix NixOS 15.09 channel generation 2016-03-07 10:51:46 +01:00
Eelco Dolstra
f46cb14bb7 Handle nixpkgs-unstable channel updates 2016-03-04 20:45:35 +01:00
Eelco Dolstra
a6ab71c2f9 Update .htaccess redirects to latest NixOS ISOs 2016-03-04 20:27:15 +01:00
Eelco Dolstra
18af18f351 Fix Git path 2016-03-03 18:29:05 +01:00
Eelco Dolstra
a058826231 Record top-level store paths of each release 
We need this to do garbage collection on cache.nixos.org now that
MANIFEST is gone.
 2016-03-03 15:59:25 +01:00
Eelco Dolstra
f7beae78eb Rewrite mirror-nixos-branch in perl; handle new Hydra situation 
We're no longer on a machine that contains the actual store paths of
the release, so we have to fetch them.
 2016-03-01 20:03:10 +01:00
Eelco Dolstra
4de7a4a85c Remove tabs 2016-03-01 11:12:36 +01:00
Eelco Dolstra
0c75ea5ba5 Push nixpkgs-unstable branch to nixpkgs-channels 
Fixes https://github.com/NixOS/nixpkgs/issues/11427.
 2015-12-04 18:15:39 +01:00
Eelco Dolstra
5fd290852c Do a git update before doing rev-parse 2015-12-02 11:42:49 +01:00
Eelco Dolstra
97b39b07dc Check for errors when compressing NARs 
Otherwise, an error (like disk full) will cause a truncated NAR file
to be uploaded to the binary cache.
 2015-10-06 23:13:49 +02:00
Eelco Dolstra
e4b04c877f Package database: Support multiple outputs 2015-09-02 18:36:17 +02:00
Eelco Dolstra
0cd5fe7808 Prefer shorter attribute paths in the package database 
E.g. "python" is better than "python27Packages.python".
 2015-09-02 18:06:27 +02:00
Eelco Dolstra
bc1b2301df Use attribute paths in the package database 2015-09-02 17:39:05 +02:00
Eelco Dolstra
d528d56e2d Prevent concurrent access to the Git clone 2015-09-02 11:31:13 +02:00
Eelco Dolstra
a441c018e2 Ensure that the channel doesn't go back in time 
This can happen when builds are restarted, causing the "latest
finished evaluation" to point to an older evaluation.
 2015-09-02 11:16:51 +02:00
Eelco Dolstra
87ccf9de70 Fix updating releases/.htaccesss 
Fixes https://github.com/NixOS/nixpkgs/issues/9228.
 2015-08-13 14:17:25 +02:00
Eelco Dolstra
c76fc42d03 Script for signing existing .narinfo files in a binary cache 2015-06-03 16:23:36 +02:00
Eelco Dolstra
d84441483e Scripts for deleting garbage from the binary cache 2015-06-03 16:23:16 +02:00
Eelco Dolstra
bc8cce06a5 After updating the channel, update/push a corresponding Git branch 
https://github.com/NixOS/nixpkgs/issues/3609
 2015-03-05 17:40:48 +01:00