When a virtual machine is provisioned to become a node in a cluster, Jetpack is automatically installed when using Azure CycleCloud. Jetpack is required on every node of a cluster, as it provides three main functions:
- Node Configuration
Azure CycleCloud uses Chef to automate the configuration of a provisioned VM into a working cluster node. A Chef client is embedded within Jetpack, and the Chef cookbooks needed for the configuration of the node are downloaded by Jetpack.
- Distributed Synchronization
Jetpack manages communication between the node and the Azure CycleCloud application server, which enables CycleCloud to monitor the status of the provisioning VMs and synchronize the orchestration of multiple nodes in the cluster.
Jetpack runs HealthCheck on VMs, and terminates them if they are unhealthy.
When you first start a cluster using Azure CycleCloud, the software fetches the jetpack installer and caches it into a locker in your Azure Storage Account. When the VMs in the cluster are provisioned, a custom script extension is executed as part of the boot process. This will download the Jetpack installer from your locker cache and install it on the VM.
Jetpack Install Location
All Jetpack files are installed inside a singular directory tree:
In addition to creating this directory, the Jetpack installer:
- Creates system init startup scripts which configure a VM as a cluster node
- Creates udev rules on Linux
- Installs the HealthCheck service
- Sets the environment variable
||Useful binaries and scripts.|
||User defined and cluster defined configuration files and scripts.|
||Logs generated by joining a cluster and converging the node, of particular interest is the chef-client.log which contains the results from converging Chef recipes.|
||Internal files. We don't recommend directly using or accessing any files in this directory as they will change significantly from release to release.|
The HealthCheck service executes user-defined scripts to determine a VM's
current viability as a cluster node. Scripts may be written in Python or be
Bash (Linux) or batch (Windows) scripts. If the script exits with a code of 254,
the VM will be terminated. Other non-zero return codes are treated as
unknown and are logged to CycleCloud. Checks are stored in
/opt/cycle/jetpack/config/healthcheck.d on Linux and in
C:\cycle\jetpack\config\healthcheck.d on Windows.
To delay a HealthCheck related termination, use the
jetpack keepalive command.
Jetpack Command Line Tool
The Jetpack command-line tool, located at
C:\cycle\jetpack\bin\jetpack, provides a useful set of subcommands for
manipulating the current VM and interacting with Azure CycleCloud.
||Retrieve a configuration value.|
||Execute a Chef converge.|
||Delay system termination by the HealthCheck Service.|
||Log a message to CycleCloud cluster UI.|
||Send an arbitrary AMQP message to the CycleCloud server.|
jetpack config is used to fetch information passed into a VM by Azure
CycleCloud. It exposes all the system properties made available via
Ohai, a subset of the VM's Azure metadata, and information about the parent CycleCloud cluster.
jetpack converge downloads all CycleCloud Project (TODO: include link to CycleCloud
project) Chef cookbooks and cluster-init scripts associated with the node, and
starts a Chef converge process that runs all the Chef recipes and cluster-init
scripts for the node.
jetpack keepalive interacts with the HealthCheck service to delay the termination of
the VM due to a failing HealthCheck. Termination can be delayed for a fixed
period or indefinitely. By default, termination is delayed for one hour.
To delay system termination by one hour:
To disable the HealthCheck service entirely, i.e. delay termination indefinitely:
jetpack keepalive forever
To delay system termination by six hours:
jetpack keepalive 6h
jetpack log sends a log message back to CycleCloud. The message will appear in
application server log (typically
main event log, and the Cluster UI page.
Each message has two properties: level and priority.
The level property indicates the type of message. Valid levels are 'info', 'warn', and 'error'. The level does not indicate importance of a given message - for example, some errors are trivial and some informational messages critical.
Priority indicates the importance of the message. Valid priority values are 'low', 'medium', and 'high'. Only messages with a priority of medium or higher are displayed on the Cluster UI page to avoid flooding the page with low priority messages.
To send an informational log message that will appear on the Cluster UI page:
jetpack log 'system is now ready'
To send a low priority log message that you do not want to appear on the Cluster UI page:
jetpack log 'system is now ready' --priority low
By default, messages with a level of error have a high priority. To send an error message:
jetpack log 'the machine cannot process jobs' --level error
To send a trivial error message:
jetpack log 'the machine cannot process jobs' --level error --priority low
jetpack send is an advanced command that you can use to send an arbitrary AMQP message
to CycleCloud. It is most useful if you have created a CycleCloud plugin that
can process that information.
You can send arbitrary strings or files with specified AMQP routing keys.