danderson/channel-scripts
0
0
Fork 0
forked from the-distro/channel-scripts
Code Pull requests Activity
183 commits 2 branches 0 tags 295 KiB
Commit graph

183 commits

This branch
This branch
All branches
Author SHA1 Message Date
Eelco Dolstra 2f7994e3de
Remove bad check 2017-07-07 19:42:20 +02:00
Eelco Dolstra b6c049019a
mirror-nixos-branch.pl: Call index-debuginfo 2017-07-07 17:12:25 +02:00
Eelco Dolstra fcd6250fd0
mirror-channel.pl: Remove 
This is no longer used, since Hydra uploads directly to S3 now. Also,
"nix copy" can copy to S3 now.
 2017-07-07 17:06:21 +02:00
Eelco Dolstra a5e125330c
generate-programs-index: Check whether symlink targets exist 
Fixes https://github.com/NixOS/nixpkgs/issues/26661
 2017-07-07 16:45:45 +02:00
Eelco Dolstra 74c917454c
Add a command for indexing debug info files by build ID 
For every file "lib/debug/.build-id/<build-id>.debug" in every store
path ending in -debug in a release, this command creates an S3 key
"debuginfo/<build-id>" specifying the location of the NAR that
contains the debug info file for that build ID. Thus,
https://cache.nixos.org/debuginfo allows debug info files to be looked
up by build ID. This allows debug info files to be fetched
automatically by dwarffs [1].

One minor issue is that in theory, multiple store paths could provide
debug info files for a particular build ID. So then when we garbage
collect a store path from the binary cache, we migth invalidate a
debug info link even though there are non-GC'ed store paths that
provide the file. Ah well.

I could have used HTTP redirects, but they're kind of a pain to
configure with S3.

[1] https://github.com/edolstra/dwarffs
 2017-07-07 15:33:27 +02:00
Eelco Dolstra a91fb566a9
Factor out the file cache 2017-07-06 14:53:52 +02:00
Eelco Dolstra 3daca6b592
Update generate-programs-index to handle .ls 2017-04-18 16:41:11 +02:00
Eelco Dolstra dd0d2d2ca6
Update build 2017-04-18 15:26:38 +02:00
Eelco Dolstra a5368b6239 Merge pull request #12 from primeos/master 
Fix the displayed hash name (SHA-512 -> SHA-256)
 2017-03-31 12:04:05 +02:00
Michael Weiss c3e8e3badb Fix the displayed hash name (SHA-512 -> SHA-256) 
The HTML pages for each channel are showing the SHA-256 hash of each
file but the table header actually referred to it as an "SHA-512 hash".
 2017-03-31 06:39:24 +02:00
Eelco Dolstra 1c07d6dd08
Lock the Git repo 2016-12-05 11:50:17 +01:00
Eelco Dolstra 2fd4d6beea Vacuum programs.sqlite 
This cuts a few hundred KiB from the database.
 2016-11-10 18:27:04 +01:00
Eelco Dolstra 6883275653 Ignore package listings with malformed UTF-8 
For example, https://cache.nixos.org/5q3s4vfwwd9n9499yg11ydziwxgn644m.ls.xz
contains the string "bokm<E5>l.alias". This causes nlohmann::json to
barf.
 2016-11-10 18:10:08 +01:00
Eelco Dolstra 2176c939fc Compute FS closures in parallel 
This is much faster.
 2016-11-10 18:10:08 +01:00
Eelco Dolstra b74e03a870 Revert "Disable programs.sqlite generation" 
This reverts commit 7f2bbc884c.
 2016-11-10 18:10:08 +01:00
Eelco Dolstra ba19471e5b Use the .ls.xz listings generated by Hydra to speed up building programs.sqlite 
This reduces the time to build the index for the regular (non-small) channel from ~10 hours to ~4 minutes.

Fixes #9.
 2016-11-10 18:10:03 +01:00
Eelco Dolstra 6391c4dcec Fix check for release existance 
We no longer have a github-link file, so this check always failed. As
a result, we were constantly re-uploading releases.
 2016-10-21 20:10:53 +02:00
Eelco Dolstra 038c8ee8d4 Update the channels repo early 
This way, users of the nixpkgs-channels repo don't have to wait for
all the slow stuff to finish (like index generation and copying of
ISOs).
 2016-10-21 19:22:05 +02:00
Eelco Dolstra 7f2bbc884c Disable programs.sqlite generation 2016-10-21 18:27:28 +02:00
Eelco Dolstra eea0729481 Generate a simple directory listing 2016-09-29 11:50:14 +02:00
Eelco Dolstra 6cb926311e Only update the release link if the release has changed 2016-09-29 10:55:11 +02:00
Eelco Dolstra fcf5a77736 Add an empty file at the release prefix for nix-channel compatibility 
nix-channel expects the prefix to exist (not return a 404).
 2016-09-26 19:50:36 +02:00
Eelco Dolstra d3d33d5b25 Store releases in S3 
Fixes #7.
 2016-09-26 18:06:51 +02:00
Eelco Dolstra becf1f2ae3 Atomically update channels/.htaccess 
Otherwise we might clobber the redirects if the disk is full, and in
any case there is a time window in which .htaccess is in an
inconsistent state.

Fixes https://github.com/NixOS/nixpkgs/issues/18544.
 2016-09-12 23:20:12 +02:00
Eelco Dolstra 8a0ed20e6d Improve error message 2016-08-22 17:26:27 +02:00
Eelco Dolstra 76ca1da3ee Add $out/bin to $PATH 2016-08-22 17:18:44 +02:00
Eelco Dolstra c981c37438 Add README 2016-08-22 17:18:29 +02:00
Eelco Dolstra ab086b22ef Revive programs.sqlite generation 
Fixes #4.
 2016-08-11 14:41:26 +02:00
Eelco Dolstra 8f7fbdaff7 Die tabs die 2016-08-08 17:32:24 +02:00
Eelco Dolstra 72501d545d Remove obsolete binary patch generation scripts 2016-04-13 11:42:52 +02:00
Eelco Dolstra 852e1b6bf2 Drop 32-bit images (except iso-minimal) 2016-03-17 12:21:13 +01:00
Eelco Dolstra 000a9f2766 Workaround for https://github.com/NixOS/nix/issues/792 2016-03-07 11:12:24 +01:00
Eelco Dolstra 373d8b7b38 Fix NixOS 15.09 channel generation 2016-03-07 10:51:46 +01:00
Eelco Dolstra f46cb14bb7 Handle nixpkgs-unstable channel updates 2016-03-04 20:45:35 +01:00
Eelco Dolstra a6ab71c2f9 Update .htaccess redirects to latest NixOS ISOs 2016-03-04 20:27:15 +01:00
Eelco Dolstra 18af18f351 Fix Git path 2016-03-03 18:29:05 +01:00
Eelco Dolstra a058826231 Record top-level store paths of each release 
We need this to do garbage collection on cache.nixos.org now that
MANIFEST is gone.
 2016-03-03 15:59:25 +01:00
Eelco Dolstra f7beae78eb Rewrite mirror-nixos-branch in perl; handle new Hydra situation 
We're no longer on a machine that contains the actual store paths of
the release, so we have to fetch them.
 2016-03-01 20:03:10 +01:00
Eelco Dolstra 4de7a4a85c Remove tabs 2016-03-01 11:12:36 +01:00
Eelco Dolstra 0c75ea5ba5 Push nixpkgs-unstable branch to nixpkgs-channels 
Fixes https://github.com/NixOS/nixpkgs/issues/11427.
 2015-12-04 18:15:39 +01:00
Eelco Dolstra 5fd290852c Do a git update before doing rev-parse 2015-12-02 11:42:49 +01:00
Eelco Dolstra 97b39b07dc Check for errors when compressing NARs 
Otherwise, an error (like disk full) will cause a truncated NAR file
to be uploaded to the binary cache.
 2015-10-06 23:13:49 +02:00
Eelco Dolstra e4b04c877f Package database: Support multiple outputs 2015-09-02 18:36:17 +02:00
Eelco Dolstra 0cd5fe7808 Prefer shorter attribute paths in the package database 
E.g. "python" is better than "python27Packages.python".
 2015-09-02 18:06:27 +02:00
Eelco Dolstra bc1b2301df Use attribute paths in the package database 2015-09-02 17:39:05 +02:00
Eelco Dolstra d528d56e2d Prevent concurrent access to the Git clone 2015-09-02 11:31:13 +02:00
Eelco Dolstra a441c018e2 Ensure that the channel doesn't go back in time 
This can happen when builds are restarted, causing the "latest
finished evaluation" to point to an older evaluation.
 2015-09-02 11:16:51 +02:00
Eelco Dolstra 87ccf9de70 Fix updating releases/.htaccesss 
Fixes https://github.com/NixOS/nixpkgs/issues/9228.
 2015-08-13 14:17:25 +02:00
Eelco Dolstra c76fc42d03 Script for signing existing .narinfo files in a binary cache 2015-06-03 16:23:36 +02:00
Eelco Dolstra d84441483e Scripts for deleting garbage from the binary cache 2015-06-03 16:23:16 +02:00