How-to: Install a Pineapple agent on Linux using SSH
Overview
This example illustrates how the SSH plugin can be used to install a Pineapple agent on Linux (specifically the Linux variants Centos and Red Hat).
Part of the default configuration
This example named ssh-007-install-pineapple-agent-linux64, including all configuration files, is included in the default configuration which is created by Pineapple, so there is no need to create it by hand.
Pineapple agents
Pineapple supports installation of agents on remote hosts. The purpose of agents is to support the all functionality of Pineapple in a distributed setting. The page Pineapple agent architecture overview describes the most important features of the agent architecture:
- The Pineapple web application (WAR archive) contains the functionality for both agents and the controlling Pineapple instance (e.g. the master).
- The stand alone web client is recommended to use as a basis for agents. It is easier to distribute and configure because it comes with its own embedded web server (Jetty).
- The prerequisite for installation and execution of an agent is the presence of Java on the host where the agent is installed. Pineapple and Jetty are Java applications.
The installation
The installation consists of three parts:
- Distribution of the Pineapple binary (e.g a ZIP archive), Pineapple configuration files and a service script to install the agent as a OS service.
- Remote creation of users and directories used for the execution of the agent.
- Installation and start of the agent as a OS service.
The installation is implemented in the module ssh-007-install-pineapple-agent-linux64 which uses the SSH plugin to do the remote installation.
Requirements for installation of the agent
The requirements for successful installation of the agent are:
- unzip must be installed in the operating system since Pineapple is distributed as a ZIP archive file. Execution of the example module ssh-002-install-yum-packages will install unzip amongst other packages.
- Java must be installed on the host since Pineapple is a Java application. Execution of the example module ssh-005-install-jvm-rpm-linux64 will install Java.
- The agent runtime user used to execute the agent must be present on the host. The runtime user is created as part of the installation described in this example.
- The runtime directory for the agent must be present on the host. The runtime directory is created as of the installation described in this example.
Instead of executing each modules manually, take a look a the example module composite-execution-003-install-pineapple-agent which is a composite that installs the three modules in sequence (i.e. ssh-002-install-yum-packages, ssh-005-install-jvm-rpm-linux64 and ssh-007-install-pineapple-agent-linux64).
User and directory setup in this example
The design of the used directories and users can be implemented in many different forms on Linux, often determined by your security requirements. The setup used in this example has the characteristics:
Target environment
The linux-vagrant environment will be used as target environment. The environment is an example environment included with Pineapple as part of the default configuration. The environment defines a network with three hosts:
- Node1 with IP Address: 192.168.34.10
- Node2 with IP Address: 192.168.34.11
- Node3 with IP Address: 192.168.34.12
The environment consist of three server nodes which can fairly easily be configured using Vagrant. Vagrant is used for convenience in the examples, since it is a easy way to create servers.
SSH installation user
Pineapple will use SSH to access the three vagrant boxes (e.g. virtual server instances) through usage of the "default" user vagrant. The vagrant user is one of the nice conveniences of using Vagrant. The vagrant user is configured to allow for sudo without the requirement for entering a password for privileged administrative commands. As a consequence of the privileges of the vagrant user (e.g. sudo with no password) the SSH commands used to install the agent will prefixed with sudo.
Agent runtime user
The agent is installed and run as a OS service. The OS service must run under a user and an appropriate one must be selected. Several options are possible:
- root. Just mentioned for completion. Not really a good option, since we want to run out software with less than root privileges.
- pineapple. Creation of a dedicated Pineapple user, completed with the required privileges. This choice make sense if Pineapple is used for infrastructure testing.
Installation using the pineapple user is implemented in this example.
Agent runtime directories
The directory setup in this example has the characteristics:
- The agent binaries will be installed in the /opt/pineapple directory.
- The agent configuration files will be installed in the /home/pineapple/ directory where the Pineapple home directory, e.g. /home/pineapple/.pineapple will created when the agent is started.
- The service script will be installed in the etc/init.d directory.
- The service script will log its output to /var/log/pineapple/pineapple-service.out. The logging is configured in the service script described below.
For the pineapple user to be able to run the agent, then directory for the agent binaries and the Pineapple home directory are created with the access mask 775:
- rwx for user pineapple.
- rwx for group members, e.g. members of the group pineapple.
- r-x for others
Master environment configuration for installation
The agents are installed using Pineapple. To install the agents at the targeted hosts, then Pineapple must be configured to know about the hosts. The knowledge about the target hosts is defined in the Pineapple environment configuration at the Pineapple instance which will used for the installation. The Pineapple instance which is used to install and control the agents play the role of master in the agent architecture. This aspect is illustrated in the figure above, where the environment configuration defined on the master Pineapple instance on the left side of the figure is used as basis for the installation.
The environment configuration at the master consists of definition of resource and credentials.
Define resources
A resource in Pineapple defines the target for some operation. In this case, to be able to distribute the agent to the three hosts defined in the linux-vagrant environment, then a resource must be defined for each host. Three resources are defined within the linux-vagrant environment in the resources file located at ${user.home}/.pineapple/conf/resources.xml:
<?xml version="1.0" encoding="UTF-8"?>
<configuration xmlns="http://pineapple.dev.java.net/ns/environment_1_0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://pineapple.dev.java.net/ns/environment_1_0
http://pineapple.dev.java.net/ns/environment_1_0.xsd">
<environments>
<environment description="Vagrant multi-machine Linux environment" id="linux-vagrant">
<resources>
<resource plugin-id="com.alpha.pineapple.plugin.ssh" credential-id-ref="ssh-node1" id="ssh-node1" >
<property value="192.168.34.10" key="host"/>
<property value="22" key="port"/>
<property value="1000" key="timeout"/>
</resource>
<resource plugin-id="com.alpha.pineapple.plugin.ssh" credential-id-ref="ssh-node2" id="ssh-node2" >
<property value="192.168.34.11" key="host"/>
<property value="22" key="port"/>
<property value="1000" key="timeout"/>
</resource>
<resource plugin-id="com.alpha.pineapple.plugin.ssh" credential-id-ref="ssh-node3" id="ssh-node3" >
<property value="192.168.34.12" key="host"/>
<property value="22" key="port"/>
<property value="1000" key="timeout"/>
</resource>
</resources>
</environment>
</environments>
</configuration>
Each resource is defined with a different ID and IP address. All other properties are identical. The role of the attribute plugin-id is to bind the resource definition to the plugin code at runtime which implements the SSH plugin.
Define credentials
Next up, is the definition of the SSH installation user which is used for SSH authentication when a session established with a SSH server. Each resource defines a reference to a credential through the credential-id-ref attribute. A credential defines the user name and password used for authentication when a SSH session is created to a host.
Three credentials are created within the linux-vagrant environment to support authentication. The credentials are defined in the credentials file located at ${user.home}/.pineapple/conf/credentials.xml:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration xmlns="http://pineapple.dev.java.net/ns/environment_1_0">
<environments>
<environment description="Vagrant multi-machine Linux environment" id="linux-vagrant">
<credentials>
<credential password="vagrant" user="vagrant" id="ssh-node1"/>
<credential password="vagrant" user="vagrant" id="ssh-node2"/>
<credential password="vagrant" user="vagrant" id="ssh-node3"/>
</credentials>
</environment>
</environments>
</configuration>
Define module
A Pineapple module is defined, which implements the installation. The core elements of a module is the model file and any binary content required for the module:
The binary content
The content directory (named bin) will contain these artifacts to support the installation:
- The Pinapple binary archive, e.g. pineapple-standalone-web-client-VERSION.zip
- The Pineapple environment configuration files (e.g. resources.xml and credentials.xml) to be used locally by the Pineapple agent.
The module is located at ${user.home}/.pineapple/modules/ssh-007-install-pineapple-agent-linux64/models and it has the structure:
ssh-007-install-pineapple-agent-linux64 | +--- models | +--- linux-vagrant.xml +--- bin +--- pineapple-standalone-web-client-VERSION.zip.DOWNLOAD-ME +--- credentials.xml +--- resources.xml
Used scripts from the Pineapple binaries
The module uses several scripts from within the main Pineapple binaries:
- The service script (PINEAPPLE_INSTALL_DIR/bin/pineapple-service.sh) is installed as a daemon at /etc/init.d/pineapple.
- The service installer script (PINEAPPLE_INSTALL_DIR/bin/install-service.sh) is used to install the service script
Add Pineapple
Add the Pineapple standalone client pineapple-standalone-web-client-VERSION.zip to ${user.home}/.pineapple/modules/ssh-007-install-pineapple-agent-linux64/bin directory.
Define model
The installation steps are implemented in the model file (please refer to steps 3 to 7 in the Install Pinapple Agent figure above). In the models directory, a model file named linux-vagrant is defined with the content:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<mmd:models xmlns:mmd="http://pineapple.dev.java.net/ns/module_model_1_0"
xmlns:shp="http://pineapple.dev.java.net/ns/plugin/ssh_1_0"
continue="false">
<mmd:variables>
<mmd:variable key="pineapple.archive" value="pineapple-standalone-web-client-1.8.0.zip" />
<mmd:variable key="pineapple.tmp.installation.dir" value="/tmp/pineapple-standalone-web-client-1.8.0" />
<mmd:variable key="pineapple.base.dir" value="/opt" />
<mmd:variable key="pineapple.installation.dir" value="/opt/pineapple" />
<mmd:variable key="pineapple.home.dir" value="/var/pineapple" />
<mmd:variable key="pineapple.user" value="pineapple" />
<mmd:variable key="pineapple.group" value="pineapple" />
</mmd:variables>
<mmd:model target-resource="regex:ssh-node.*" target-operation="deploy-configuration" description="Install binaries" >
<mmd:content>
<shp:ssh>
<shp:copy-to source="modulepath:bin/${pineapple.archive}" destination="/tmp/${pineapple.archive}" substitute-variables="false"/>
<shp:execute command="sudo unzip /tmp/${pineapple.archive} -d /tmp" />
<shp:execute command="sudo rm -rf /tmp/${pineapple.archive}" />
<shp:execute command="sudo mv -f ${pineapple.tmp.installation.dir} ${pineapple.installation.dir}" />
<shp:execute command="sudo chmod +x ${pineapple.installation.dir}/bin/setup.sh" />
<shp:execute command="sudo ${pineapple.installation.dir}/bin/setup.sh" />
<shp:execute command="sudo chown -R ${pineapple.user}:${pineapple.group} ${pineapple.installation.dir}" />
<shp:execute command="sudo chmod -R 775 ${pineapple.installation.dir}" />
</shp:ssh>
</mmd:content>
</mmd:model>
<mmd:model target-resource="regex:ssh-node.*" target-operation="{test,deploy-configuration}" description="Test binaries installation" >
<mmd:content>
<shp:ssh>
<shp:assert-contains command="getent passwd ${pineapple.group}" expected-value="pineapple:x" /> <!-- Test user exists -->
<shp:assert-contains command="getent group ${pineapple.group}" expected-value="pineapple:x" /> <!-- Test group exists -->
<shp:assert-contains command="id -gn ${pineapple.user}" expected-value="${pineapple.group}" /> <!-- Test primary group for pineapple.user -->
<shp:assert-equals command="test -d ${pineapple.base.dir} && echo "Found"|| echo "NotFound"" expected-value="Found" /> <!-- Test dir exist -->
<shp:assert-equals command="test -d ${pineapple.installation.dir} && echo "Found"|| echo "NotFound"" expected-value="Found" /> <!-- Test dir exist -->
<shp:assert-equals command="stat --format "%a" ${pineapple.installation.dir}" expected-value="775"/> <!--test directory permissions -->
</shp:ssh>
</mmd:content>
</mmd:model>
<mmd:model target-resource="regex:ssh-node.*" target-operation="deploy-configuration" description="Install configuration files" >
<mmd:content>
<shp:ssh>
<shp:execute command="sudo ${pineapple.installation.dir}/bin/create-runtime-dirs.sh" />
<shp:copy-to source="modulepath:bin/resources.xml" destination="/tmp/resources.xml"/>
<shp:copy-to source="modulepath:bin/credentials.xml" destination="/tmp/credentials.xml"/>
<shp:execute command="sudo cp /tmp/resources.xml ${pineapple.home.dir}/conf/resources.xml" />
<shp:execute command="sudo cp /tmp/credentials.xml ${pineapple.home.dir}/conf/credentials.xml" />
<shp:execute command="sudo chown ${pineapple.group}:${pineapple.user} ${pineapple.home.dir}/conf/resources.xml" />
<shp:execute command="sudo chown ${pineapple.group}:${pineapple.user} ${pineapple.home.dir}/conf/credentials.xml" />
<shp:execute command="sudo chmod 644 ${pineapple.home.dir}/conf/resources.xml" />
<shp:execute command="sudo chmod 644 ${pineapple.home.dir}/conf/credentials.xml" />
</shp:ssh>
</mmd:content>
</mmd:model>
<mmd:model target-resource="regex:ssh-node.*" target-operation="{test,deploy-configuration}" description="Test configuration installation" >
<mmd:content>
<shp:ssh>
<shp:assert-equals command="test -d ${pineapple.home.dir} && echo "Found"|| echo "NotFound"" expected-value="Found" /> <!-- Test dir exist -->
<shp:assert-equals command="stat --format "%a" ${pineapple.home.dir}" expected-value="775"/> <!--test directory permissions -->
<shp:assert-equals command="test -d ${pineapple.home.dir}/conf && echo "Found"|| echo "NotFound"" expected-value="Found" /> <!-- Test dir exist -->
<shp:assert-equals command="stat --format "%a" ${pineapple.home.dir}/conf" expected-value="775"/> <!--test directory permissions -->
<shp:assert-equals command="test -d ${pineapple.home.dir}/modules && echo "Found"|| echo "NotFound"" expected-value="Found" /> <!-- Test dir exist -->
<shp:assert-equals command="stat --format "%a" ${pineapple.home.dir}/modules" expected-value="775"/> <!--test directory permissions -->
<shp:assert-equals command="test -d ${pineapple.home.dir}/reports && echo "Found"|| echo "NotFound"" expected-value="Found" /> <!-- Test dir exist -->
<shp:assert-equals command="stat --format "%a" ${pineapple.home.dir}/reports" expected-value="775"/> <!--test directory permissions -->
<shp:assert-equals command="test -f ${pineapple.home.dir}/conf/resources.xml && echo "Found" || echo "NotFound"" expected-value="Found" />
<shp:assert-equals command="stat --format "%a" ${pineapple.home.dir}/conf/resources.xml" expected-value="644"/> <!--test directory permissions -->
<shp:assert-equals command="test -f ${pineapple.home.dir}/conf/credentials.xml && echo "Found" || echo "NotFound"" expected-value="Found" />
<shp:assert-equals command="stat --format "%a" ${pineapple.home.dir}/conf/resources.xml" expected-value="644"/> <!--test directory permissions -->
<shp:assert-equals command="stat --format "%U" ${pineapple.home.dir}/conf/resources.xml" expected-value="${pineapple.user}"/> <!--test directory owner -->
<shp:assert-equals command="stat --format "%G" ${pineapple.home.dir}/conf/resources.xml" expected-value="${pineapple.group}"/> <!--test directory group -->
<shp:assert-equals command="stat --format "%a" ${pineapple.home.dir}/conf/credentials.xml" expected-value="644"/> <!--test directory permissions -->
<shp:assert-equals command="stat --format "%U" ${pineapple.home.dir}/conf/credentials.xml" expected-value="${pineapple.user}"/> <!--test directory owner -->
<shp:assert-equals command="stat --format "%G" ${pineapple.home.dir}/conf/credentials.xml" expected-value="${pineapple.group}"/> <!--test directory group -->
</shp:ssh>
</mmd:content>
</mmd:model>
<mmd:model target-resource="regex:ssh-node.*" target-operation="deploy-configuration" description="Install service" >
<mmd:content>
<shp:ssh>
<shp:execute command="sudo ${pineapple.installation.dir}/bin/install-service.sh" />
</shp:ssh>
</mmd:content>
</mmd:model>
<mmd:model target-resource="regex:ssh-node.*" target-operation="{test,deploy-configuration}" description="Test service intallation" >
<mmd:content>
<shp:ssh>
<shp:assert-contains command="sudo /sbin/service pineapple status" expected-value="is running" />
</shp:ssh>
</mmd:content>
</mmd:model>
<mmd:model target-resource="regex:ssh-node.*" target-operation="undeploy-configuration" description="Uninstall Pineapple" >
<mmd:content>
<shp:ssh>
<shp:execute command="sudo ${pineapple.installation.dir}/bin/uninstall-service.sh" />
<shp:execute command="sudo rm -rf /tmp/${pineapple.archive}" />
<shp:execute command="sudo rm -rf ${pineapple.installation.dir}" />
<shp:execute command="sudo rm -rf ${pineapple.home.dir}" />
<shp:execute command="sudo userdel ${pineapple.user}" />
</shp:ssh>
</mmd:content>
</mmd:model>
</mmd:models>
The configuration details
Two schema are used in the model file. The http://pineapple.dev.java.net/ns/module_model_1_0 is used to define the namespace mmd which defines the general infrastructure for models. The http://pineapple.dev.java.net/ns/plugin/ssh_1_0 schema is used to define the namespace shp which is used to define the model for the SSH plugin. Since multiple schemas are used to define the model file, the elements are qualified.
The target-resource attribute defines a reference to the resource which is targeted when the model executed. In this case, the value regex:ssh-node.* defines a regular expression for targeting multiple resources, e.g. all the resources starting with ssh-node.
The main part of the model consists of sub models, which performs the installation steps:
- Copy and unzip the Pineapple binaries.
- Test the installation of the binaries.
- Create the the runtime directories used by Pineapple, copy the environment configuration configuration files into the runtime directory structure.
- Test the configuration.
- Install the OS service.
- Test that the OS service started the Pineapple agent successfully.
Please notice: The environment variable JAVA_HOME isn't set as a result of the installation.
Invoke Pineapple to execute model
Start your Pineapple client of choice:
- Select the modules named ssh-007-install-pineapple-agent-linux64
- Select the linux-vagrant model.
- Invoke the deploy-configuration operation to install the Pineapple agent at the three nodes defined in the linux-vagrant environment.