Cluster Configuration
Configuration objects are rank 3, and subordinate to node and nodearray. The configuration object define the configurable properties for the configuration code that runs on the nodes and CycleCloud Project configurations.
Object Attributes
Configuration object attributes behave like other objects, but are extended to provide nested definitions. Internally, the configuration sections are merged, so there can be an arbitrary number of sections.
No quotes are necessary for strings or for boolean expressions, true or false.
[[[configuraton my-project]]]
Attribute1 = Value1
Attribute2 = Value2
KeyAttribute3.Attribute3 = true
[[[configuraton cyclecloud.mounts.mount1]]]
Attribute1 = Value1
Supported Configurations
CycleCloud supports a number of default configuration objects. These supported objects are contained under the name cyclecloud.
[[[configuration cyclecloud]]]
CycleCloud supports the parameterized configuration of many system services.
| Attribute | Type | Description |
|---|---|---|
| maintenance_converge.enabled | Boolean | CycleCloud nodes are reconfigured every 20 minutes to ensure they are in the correct state. There are times when you may not want this to be the default behavior such as when you are manually testing and updating the configuration on a node. Setting this value to false will make the node configure itself only once. Default: true |
| node.prevent_metadata_access | Boolean | Prevents users, other than the root user or cyclecloud user from accessing the VM metadata from the node. These access rules are applied in iptables. Default: true |
| timezone | String | The timezone for a node can be changed by setting this attribute to any valid timezone string, for example PST, EST. Default: UTC |
| ntp.disabled | Boolean | Opt-out of ntp time service by setting true. Default: false |
| ntp.servers | List (String) | A list of NTP servers to use. Default: pool.ntp.org |
| keepalive.timeout | Integer | The amount of time in seconds to keep a node "alive" if it has not finished installing/configuring software. Default: 14400 (4 hours) |
| discoverable | Boolean | Whether or not this node can be "discovered" (searched for) by other nodes started by CycleCloud. Default: false |
| autoscale.forced_shutdown_timeout | Integer | The amount of time (in minutes) before a forced shutdown occurs if autoscale cannot scale the node down successfully. Default: 15 |
| security.limits | Integer | Linux only. The limits to apply to the node. Domain, type, and item can be specified for any valid value defined. Defaults: security.limits.\*.hard.nofile = 524288 and security.limits.\*.soft.nofile = 1048576 |
| mounts | Nested | For NFS exporting and mounting and volume mounting. |
| selinux.policy | String | Linux only. Bypass an enforced selinux policy for custom images. Already disabled on core CycleCloud images. |
| install_epel | Boolean | Add the extended packages repo for yum on RedHat variant image. Default: true |
| disable_rhui | Boolean | Opt-out of Red Hat repository configs. Default : false |
| ganglia.install | Boolean | Opt-out of ganglia installation by setting false. Default: true |
| fail2ban.enabled | Boolean | Opt-out of fail2ban installation by setting false. Default: true |
| dns.domain | String | Use nsupdate to force a dynamic DNS record update. Useful ONLY when allowed by DNS policy, and the cluster is using a DNS server that allows dynamic updates. Default: nil |
| dns.alias | String | Use nsupdate to force a dynamic DNS record update. Useful ONLY when allowed by DNS policy, and the cluster is using a DNS server that allows dynamic updates. Default: nil |
| replace_sudoers | Boolean | Allow Cyclecloud to managed the sudoers configuration. Disabling can interfere with user or scheduler services. Default: true |
| Attribute | Type | Description |
|---|---|---|
| keepalive.timeout | Integer | The amount of time in seconds to keep a node "alive" if it has not finished installing/configuring software. Default: 14400 (4 hours) |
| discoverable | Boolean | Whether or not this node can be "discovered" (searched for) by other nodes started by CycleCloud. Default: false |
| security.limits | Integer | Linux only. The limits to apply to the node. Domain, type, and item can be specified for any valid value defined. Defaults: security.limits.\*.hard.nofile = 524288 and security.limits.\*.soft.nofile = 1048576 |
| mounts | Nested | For NFS exporting and mounting and volume mounting. |
| selinux.policy | String | Linux only. Bypass an enforced selinux policy on cluster instances. Default: nil |
| dns.domain | String | Use nsupdate to force a dynamic DNS record update. Useful ONLY when allowed by DNS policy, and the cluster is using a DNS server that allows dynamic updates. Default: nil |
| dns.alias | String | Use nsupdate to force a dynamic DNS record update. Useful ONLY when allowed by DNS policy, and the cluster is using a DNS server that allows dynamic updates. Default: nil |
| samba.enabled | Boolean | Linux only. Installs Samba on a filer for use with Windows execute nodes. Default: false |
[[[configuration cyclecloud.cluster]]]
CycleCloud cluster namespace contains configurations for distributed services and clustered applications.
| Attribute | Type | Description |
|---|---|---|
| autoscale.idle_time_after_jobs | Integer | Nodes are terminated if they are idle for specified time (in seconds) after they have run jobs. Default: 1800 |
| autoscale.idle_time_before_jobs | Integer | Nodes are terminated if they are idle for specified time (in seconds) before they have run jobs. Default: 1800 |
| autoscale.stop_interval | Integer | Time delay between runs of auto-stop checks (in seconds). Default: 60. |
| autoscale.use_node_groups | Boolean | Enable grouped nodes - equivalent to placement groups. Effects only PBSPro and Grid Engine clusters. Default: true |
[[[configuration cyclecloud.hosts.standalone_dns]]]
CycleCloud will configure the /etc/hosts file to contain a large set of hosts so that forward and reverse name resolution is functional. These configurations act to operate as DNS replacement configured on the individual nodes, not centrally managed, therefore called stand-alone DNS.
| Attribute | Type | Description |
|---|---|---|
| enabled | Boolean | Enable management of the etc hosts file. Default: true. |
| alt_suffix | String | Override the default domain name of the VNET. Example: contoso.com |
| subnets | List (String) | List of CIDR blocks for extended standalone name resolution. |
By default, CycleCloud will inspect the network interface and compose the /etc/hosts file to include hosts in the subnet mask. Additional ranges can be added using the subnets attribute.
[[[configuration cyclecloud.hosts.standalone_dns]]]
alt_suffix = my-domain.local
subnets = 10.0.1.0/24, 10.0.5.0/24
To override and disable the standalone service:
[[[configuration ]]]
cyclecloud.hosts.standalone_dns.enabled = false
[[[configuration cyclecloud.mounts]]]
A significant subdomain of the cyclecloud configuration is mounts. Each named mount section corresponds to an entry in /etc/fstab.
An example of a mount section named primary.
[[[configuration cyclecloud.mounts.primary]]]
type = nfs
mountpoint = /usr/share/lsf
export_path = /mnt/raid/lsf
options = hard,proto=tcp,mountproto=tcp,retry=30,actimeo=3600,nolock
address = 10.0.0.14
Important
The mount section name correlates to the mount attribute of a [[[volume]]] object.
| Attribute | Type | Description |
|---|---|---|
| type | String | The type attribute should be set to nfs for all NFS exports to differentiate from volume mounts and other shared filesystem types. |
| export_path | String | The location of the export on the NFS filer. If an export_path is not specified, the mountpoint of the mount will be used as the export_path. |
| fs_type | String | Type of filesystem to use. E.g ext4, xfs. |
| mountpoint | String | The location where the filesystem will be mounted after any additional configuration is applied. If the directory does not already exist, it will be created. |
| cluster_name | String | The name of the CycleCloud cluster which exports the filesystem. If not set, the node's local cluster is assumed. |
| address | String | The explicit hostname or IP address of the filesystem. If not set, search will attempt to find the filesystem in a CycleCloud cluster. |
| options | String | Any non-default options to use when mounting the filesystem. |
| disabled | Boolean | If set to true, the node will not mount the filesystem. |
| raid_level | Integer | The type of RAID configuration to use when multiple devices/volumes are being used. Defaults to a value of 0, meaning RAID0, but other raid levels can be used such as 1 or 10. |
| raid_device_symlink | String | When a raid device is created, specifying this attribute will create a symbolic link to the raid device. By default, this attribute is not set and therefore no symlink is created. This should be set in cases where you need access to the underlying raid device. |
| devices | List (String) | This is a list of devices that should compose the mountpoint. In general, this parameter shouldn't be specified (as CycleCloud will set this for you based on [[[volume]]] sections), but you can manually specify the devices if so desired. |
| vg_name | String | Devices are configured on Linux using the Logical Volume Manager (LVM). The volume group name will be automatically assigned, but in cases where a specific name is used, this attribute can be set. The default is set to cyclecloud-vgX, where X is an automatically assigned number. |
| lv_name | String | Devices are configured on Linux using the Logical Volume Manager (LVM). This value is automatically assigned and does not need specification, but if you want to use a custom logical volume name, it can be specified using this attribute. Defaults to lv0. |
| order | Integer | By specifying an order, you can control the order in which mountpoints are mounted. The default order value for all mountpoints is 1000, except for 'ephemeral' which is 0 (ephemeral is always mounted first by default). You can override this behavior on a case-by-case basis as needed. |
| encryption.bits | Integer | The number of bits to use when encrypting the filesystem. Standard values are 128 or 256 bit AES encryption. This value is required if encryption is desired. |
| encryption.key | String | The encryption key to use when encrypting the filesystem. If omitted, a random 2048 bit key will be generated. The automatically generated key is useful for when you are encrypting disks that do not persist between reboots (e.g. encrypting ephemeral devices). |
| encryption.name | String | The name of the encrypted filesystem, used when saving encryption keys. Defaults to cyclecloud_cryptX, where X is an automatically generated number. |
| encryption.key_path | String | The location of the file the key will be written on disk to. Defaults to /root/cyclecloud_cryptX.key, where X is a automatically generated number. |
[[[configuration cyclecloud.exports]]]
Similar to mounts, CycleCloud nodes can be configured as NFS servers if the server recipe is enabled. Export section corresponds to /etc/exports entry.
An example of using exports with an export object named nfs_data:
[[[configuration cyclecloud.exports.nfs_data]]]
type = nfs
export_path = /mnt/exports/nfs_data
writable = false
| Attribute | Type | Description |
|---|---|---|
| type | String | REQUIRED The type attribute must be set to nfs for all NFS exports to differentiate from other shared filesystem types. |
| export_path | String | The local path to export as an NFS filesystem. If the directory does not exist already, it will be created. |
| owner | String | The user account that should own the exported directory. |
| group | String | The group of the user that should own the exported directory. |
| mode | String | The default filesystem permissions on the exported directory. |
| network | String | The network interface on which the directory is exported. Defaults to all: *. |
| sync | Boolean | Synchronous/asynchronous export option. Defaults to true. |
| writable | Boolean | The ro/rw export option for the filesystem. Defaults to true. |
| options | String | Any non-default options to use when exporting the filesystem. |
[[[configuration]]]
A limited number of configuration options are available under no namespace, usually for legacy reasons.
| Attribute | Type | Description |
|---|---|---|
| http_proxy | String | The hostname or IP address of the proxy server to use on the node (e.g. 10.10.10.10) |
| https_proxy | String | The hostname or IP address of the HTTPS proxy server to use on the node. (e.g. 10.10.0.1) |
| no_proxy | String | A comma separated list of hostnames or IP addresses not to use a proxy server for. Default: 169.254.169.254 to allow access to Azure metadata |