How to Download and Install Progress Chef 360 and Node Management for Job Orchestration

Progress Chef Courier is an enterprise job orchestration and automation platform that allows you to perform any action over your IT fleet with immense flexibility. It allows users to orchestrate timing and targeting dimensions for the action/execution capabilities that all other Chef products provide.

Chef Courier gives you complete control and flexibility over how customers can perform actions on the fleet, enabling IT operators, InfoSec engineers and other administrators to run on-demand jobs. Importantly, it removes the need to build custom solutions for emergency tasks and reactionary work, reducing the burden on your team.

Three different aspects help customers control their fleet:

Whenever: Perform the action anytime – i.e., now, on-demand, scheduled, recurring, or recurring at a specific interval with exception.

Wherever: Target a specific node or a subset of nodes. For example, start with staging, production or a region like the US-East.

Whatever: Perform any action, such as an OS command, a cookbook run, an InSpec scan, or a combination of all these. The output of one command can be passed to another command, or it can be conditional, where a command runs if the previous run succeeds.

A step-by-step guide will help you install and use Chef Courier easily.

Before that, let’s learn about the Progress Chef 360 platform.

The Chef 360 Platform

Courier is built and installed on the Chef 360 platform, a base platform where you can log in, access the dashboard, create users and manage all administrative tasks.

The vision of Chef 360: A modern, cloud-native DevOps platform that democratizes DevOps by empowering IT operators and DevOps Engineers to manage mission-critical infrastructure securely.

Chef 360 Platform Goals:

• Unified user experience and optimized user journey
• Built for on-premises, SaaS and cloud-native
• Built to handle scale
• A single platform deployed anywhere
• Enable product-led growth

How to Download and Install the Chef 360 Platform:

• Download Chef 360 installer: $ curl https://replicated.app/embedded/chef-360/stable -H "Authorization: <YOUR_AUTH_CODE>" -o chef-360-stable.tgz 

[Once you are entitled to Chef 360, you’ll receive an Auth code to install the platform.]

• Extract the chef-360.tgz file: $ tar -xvzf chef-360-stable.tgz 

• Run the installer: $ sudo ./chef-360 install --license license.yaml 

Configure Chef 360

To configure Chef 360, log into the Chef 360 Admin Console using the link provided during installation and navigate the configuration options.

Note: Point your machine IP to FQDN.

• Follow the instructions in the Admin Console to configure the following:
• Configuration Option

o Example: Advance configuration and Show Add-on's

• Primary Tenant

o Example: Tenant name: demo

o Tenant TLD: chefdemo.cloud

o Tenant subdomain: chef360

Note: The system automatically updates the port to 31000, which is the port for the API gateway.

• Tenant SMTP Settings
• Tenant Administrator
• Primary Tenant Organization
• API and UI settings
• RabbitMQ

After entering your deployment settings, verify that all settings are correct, then select ‘Continue’ to save the configuration.

Chef 360 runs preflight checks to verify the deployment. Once deployed, you should see the current configuration installed, the licensing key and a few other options.

Note: If you face any problems with deployment settings, the Admin Console returns a warning. Correct the settings and select Re-run.

Install CLIs and Register Your Workstation:

• Download and install the CLIs from the Chef 360 Admin Console
• In the Chef 360 Admin Console, select ‘Application’ from the top navigation menu, select ‘Dashboard,’ then select ‘Download CLI.’
• From the download page, follow the instructions to download and install each CLI tool.
• After the CLIs are installed, run each CLI with the --help flag to verify that it’s installed.
• Example: $ chef-courier-cli --help 

Different Types of CLIs:

• chef-platform-auth-cli: Used by admins to manage users, roles and orgs
• chef-node-management-cli: Used by practitioners to bring nodes under management
• chef-courier-cli: Used by practitioners to create and schedule courier jobs

You need to register the CLIs once you have installed them.

Registration creates a profile on the user machine containing user credentials, roles and permissions.

Registering Workstation:

• Create a profile associating your workstation with a specific tenant, organization and role.

$ chef-courier-cli register-device --device-name <WORKSTATION_NAME> --profile-name <PROFILE_NAME> --url <TENANT_URL> 

Once your workstation is registered, you should view all the details below.

Test your connection by getting the role associated with your user account.

$ chef-courier-cli user-account self get-role --profile <PROFILE_NAME> 

Node Management

You can define skills and enroll nodes while installing and managing abilities with Node Management.

You can perform the following:

• Get Visibility and Discovery – Gain visibility to the enterprise fleet and be able to discover nodes in the fleet too.
• Enrollment –Use automation capabilities to ensure that nodes are brought under the management of Chef 360
• Settings – Manage configurations required for a Chef skill
• Skill Management –Use automation capabilities to better manage agents (skills) on the nodes
• Filter Management –Arrange nodes in the fleet into Node filter and Node list

Setting Up a Node for Creating a Courier Job

• Define Skills
• Create Override Settings
• Enroll Nodes
• Install Skills

Define Skills

A skill is an agent that allows you to perform actions on a node to achieve an outcome. It can be either a canister-based or on-canister-based skill. Skills can be Chef-owned (Infra, InSpec, Courier, etc.) and managed by the customer on Node Management.

Create Skill Definitions

The following Chef 360 skills are required or recommended for managing nodes, running actions and returning action results:

• Courier Runner skill
• Gohai skill
• Shell Interpreter skill
• Restart Interpreter skill

Courier Runner Skill: The Courier Runner skill interprets the Courier Job definitions sent to it by the Courier Dispatcher and executes them on the node.

• Create a JSON file called courier-runner-skill.json containing the following skill definition for Courier Runner. For the template, follow the Chef docs here.
• Create the skill: $ chef-node-management-cli management skill create-skill --body-file courier-runner-skill.json 

You can create Gohai skills, Shell Interpreter and Restart Interpreter skills in a similar manner. Find more details here.

• Verify if all skills are created using the command: $ chef-node-management-cli management skill find-all-skills 

Skill Assembly: A skill assembly is a rule of skills that must be added to the nodes. For the first step, we will create a default/blank skill assembly and update it later to add more skills.

• Create a file named skill-assembly.json that contains the skill rules. The default template is available here.

{ 
     "name": "my-skill-assembly-with-restart", 
     "skills": [ 
         { 
         "action": "add", 
         "skill": { 
            "name": "courier-runner", 
            "channel": "stable", 
            "value": ["1.3.1"] 
            } 
         }, 
         { 
         "action": "add", 
         "skill": { 
          "channel": "stable", 
          "name": "chef-gohai", 
          "value": ["0.3.1"] 
        } 
      }, 
      { 
        "action": "add", 
        "skill": {  
         "name": "shell-interpreter", 
         "channel": "stable", 
         "value": ["0.3.1"] 
       } 
     } 
   ] 
}

• Run the command to create the skill assembly. $ chef-node-management-cli management assembly create-assembly --body-file skill-assembly.json

Settings

Create Settings: Manage configurations required for a skill using Global Defaults or Override Settings.

Global Defaults: This contains key-value pairs that a skill needs to function on a node. By default, all nodes receive settings defined in Global Defaults.

Override Settings: This contains key-value pairs that a skill would need to function on a node. They override settings specified in Global Defaults.

Create node override settings for the Courier Runner and Gohai skills.

• Create a file named node-override-setting.json and paste the following JSON into it. The template is located on this page.
• Use the create-setting subcommand to create the override setting.

o $ chef-node-management-cli management setting create-setting --body-file node-override-setting.json 


• Save the setting ID string returned above by the chef-node-management-cli.
• Verify that the setting was created.

o $ chef-node-management-cli management setting find-one-setting --settingId <SETTING_ID> 

Create a node cohort:

A node cohort provides a common set of override settings, which are required skill settings (like a config.rb for Chef Infra Client today), and a skill assembly, which are skills that are installed or removed from a node after enrollment. This set of skills and settings are applied to every node in a cohort. Nodes must be assigned a node cohort during enrollment.

• Create a file named node-cohort.json that contains the skill assembly ID and the ID of the setting it should be associated with.

• Create the node cohort with the command: $ chef-node-management-cli management cohort create-cohort --body-file node-cohort.json 

Enroll Nodes

Node enrollment brings nodes under management by Chef 360. You can enroll nodes using a JSON or YAML file.

• This allows you to Install the Chef Node Management client and required skills, which allows them to connect to Chef 360 authorized nodes for reporting and agent-based operations.
• Categorize each node’s status into different enrollment levels.

Requirements:

• Nodes must be public and should be accessible with SSH or WinRM.
• A node IP address cannot be localhost (127.0.0.1).
• A node CIDR address cannot be in the same range as the Chef 360 services. The default CIDR range for Chef 360 services is 10.244.0.0/16 or 10.96.0.0/12.
• Port 22 must be open and should have sudo privileges.

In this example, we are creating a JSON file for node management.

• Create a file name enroll-linux.json with the following payload.

Note: The SSH key can’t be a multiline string in the JSON payload. Replace new lines with the newline character \n. You can use the following command to replace new lines in a PEM file: $ awk 'NF {sub(/\r/, ""); printf "%s\n",$0;}' <FILENAME.pem> 

• To enroll the node, run the command: $ chef-node-management-cli enrollment enroll-node --body-file enroll-linux.json 

The above command will place the enrolment request and return a response like the one below, containing the node and enrolment ID.

Check the status of Node Enrollment.

• To check the enrollment status, use the nodeId received from the above step:

$ chef-node-management-cli status get-status --nodeId <NodeId> 

You can find more details about Enrolling Nodes here.

Adding and Installing Skill to Courier Node.

• Create a file named update-skill-assembly-for-courier.json with the below payload

{ 
   "skills": [ 
       { 
       "action": "add", 
       "skill": { 
           "name": "courier-runner", 
           "channel": "stable", 
           "value": ["1.3.1"] 
           } 
         }, 
         { 
         "action": "add", 
         "skill": { 
           "channel": "stable", 
           "name": "chef-gohai", 
           "value": ["0.3.1"] 
         } 
      } 
   ] 
}

• Run the command by updating the Node ID.
$ chef-node-management-cli management assembly update-assembly --skillAssemblyId --body-file update-skill-assembly-for-courier.json 
• Create a file named install-si-to-node.json with the payload below.

{ 
     "skills": [ 
        { 
        "action": "add", 
        "skill": { 
            "name": "courier-runner", 
            "channel": "stable", 
            "value": ["1.3.1"] 
            } 
        }, 
        { 
        "action": "add", 
        "skill": { 
          "channel": "stable", 
          "name": "chef-gohai", 
          "value": ["0.3.1"] 
        } 
      }, 
      { 
        "action": "add", 
        "skill": { 
          "name": "shell-interpreter", 
          "channel": "stable", 
          "value": ["0.3.1"] 
        } 
      } 
    ] 
} 

• Run the command by updating the Node ID. chef-node-management-cli management assembly update-assembly --skillAssemblyId --body-file install-si-to-node.json 

Run a Courier Job on the Node

A single place in which what, where and when are provided for actions to be performed. This umbrella term is used at the highest level for Courier. Users can define multiple jobs.

Example:

• Create a job definition by creating a file create-job-definition.json

o For job template refer here.

• Create a job run: $ chef-courier-cli scheduler jobs add-job --body-file create-job-simple.json 

• Get job run status.
• Request the details of each job run using the instance ID: $ chef-courier-cli state instance list-instance-runs --<INSTANCE_ID> 

In conclusion, please follow the steps provided in this blog to set up Chef Courier effortlessly. Chef Courier is designed to dissolve barriers between teams and help democratize DevOps. Enjoy seamless functionality and superior performance from the moment you begin using it.

We are confident this product will help you simplify your workflow orchestration tasks. Thank you for choosing our product, and welcome to a new level of efficiency and innovation.

Reference

Videos:

• Progress Chef Courier - Overview
• Progress Chef Courier Demo on Action Chaining (Whatever)
• Progress Chef Demo on Controlled Rollout (Wherever)
• Progress Chef Courier Demo on Scheduling Jobs (Whenever)

Community page: Click here.