From e8afde507979ff5fef53cb692d398c6a115859d6 Mon Sep 17 00:00:00 2001 From: Graham Christensen Date: Wed, 14 Apr 2021 15:03:46 -0400 Subject: [PATCH 1/2] Write up some documentation on notifications --- doc/manual/src/SUMMARY.md | 5 +++- doc/manual/src/notifications.md | 42 +++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 1 deletion(-) create mode 100644 doc/manual/src/notifications.md diff --git a/doc/manual/src/SUMMARY.md b/doc/manual/src/SUMMARY.md index 586c09aa..a13cc98d 100644 --- a/doc/manual/src/SUMMARY.md +++ b/doc/manual/src/SUMMARY.md @@ -5,6 +5,9 @@ - [Creating and Managing Projects](projects.md) - [Using the external API](api.md) - [Monitoring Hydra](./monitoring/README.md) + +## Developer's Guide +- [Hacking](hacking.md) +- [Hydra Notifications](notifications.md) ----------- [About](about.md) -[Hacking](hacking.md) diff --git a/doc/manual/src/notifications.md b/doc/manual/src/notifications.md new file mode 100644 index 00000000..3a47d5e9 --- /dev/null +++ b/doc/manual/src/notifications.md @@ -0,0 +1,42 @@ +# `hydra-notify` and Hydra's Notifications + +Hydra uses a notification-based subsystem to implement some features and support plugin development. Notifications are sent to `hydra-notify`, which is responsible for dispatching each notification to each plugin. + +Notifications are passed from `hydra-queue-runner` to `hydra-notify` through Postgres's `NOTIFY` and `LISTEN` feature. + +## Notification Types + +Note that the notification format is subject to change and should not be considered an API. Integrate with `hydra-notify` instead of listening directly. + +### `build_started` + +* **Payload:** Exactly one value, the ID of the build. +* **When:** Issued directly before building happens, and only if the derivation's outputs cannot be subsituted. +* **Delivery Semantics:** Ephemeral. `hydra-notify` must be running to react to this event. No record of this event is stored. + +### `step_finished` + +* **Payload:** Three values, tab separated: The ID of the build which the step is part of, the step number, and the path on disk to the log file. +* **When:** Issued directly after a step completes, regardless of success. Is not issued if the step's derivation's outputs can be substituted. +* **Delivery Semantics:** Ephemeral. `hydra-notify` must be running to react to this event. No record of this event is stored. + +### `build_finished` + +* **Payload:** At least one value, tab separated: The ID of the build which finished, followed by IDs of all of the builds which also depended upon this build. +* **When:** Issued directly after a build completes, regardless of success and substitutability. +* **Delivery Semantics:** At least once. + +`hydra-notify` will call `buildFinished` for each plugin in two ways: + +* The `builds` table's `notificationspendingsince` column stores when the build finished. On startup, `hydra-notify` will query all builds with a non-null `notificationspendingsince` value and treat each row as a received `build_finished` event. + +* Additionally, `hydra-notify` subscribes to `build_finished` events and processes them in real time. + +After processing, the row's `notificationspendingsince` column is set to null. + +## Development Notes + +### Re-sending a notification + +Notifications can be experimentally re-sent on the command line with `psql`, with `NOTIFY $notificationname '$payload'`. + From 6eb701fcf266013edd1b8860093268fd36187095 Mon Sep 17 00:00:00 2001 From: Graham Christensen Date: Thu, 17 Jun 2021 13:20:13 -0400 Subject: [PATCH 2/2] notifications: Document an example scenario where builds_finished is triggered twice. --- doc/manual/src/notifications.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/manual/src/notifications.md b/doc/manual/src/notifications.md index 3a47d5e9..1fcd3c57 100644 --- a/doc/manual/src/notifications.md +++ b/doc/manual/src/notifications.md @@ -34,6 +34,8 @@ Note that the notification format is subject to change and should not be conside After processing, the row's `notificationspendingsince` column is set to null. +It is possible for subsequent deliveries of the same `build_finished` data to imply different outcomes. For example, if the build fails, is restarted, and then succeeds. In this scenario the `build_finished` events will be delivered at least twice, once for the failure and then once for the success. + ## Development Notes ### Re-sending a notification