TOSCA

The Infrastructure Manager supports the definition of Cloud topologies using OASIS TOSCA Simple Profile in YAML Version 1.0.

The TOSCA support has been developed under de framework of the INDIGO DataCloud EU project. You can see some input examples at https://github.com/indigo-dc/tosca-types/tree/master/examples.

Basic example

This TOSCA file describes a cloud topology with 2 VM with at least 2 CPUs and 2 GB of RAM and 40 GB of root disk, connected with a public IP, using an Ubuntu 20.04 image. As outputs, the TOSCA file will return the public IP of the VM and the SSH credentials to access it:

tosca_definitions_version: tosca_simple_yaml_1_0

imports:
- indigo_custom_types: https://raw.githubusercontent.com/indigo-dc/tosca-types/master/custom_types.yaml

topology_template:

  node_templates:

    simple_node:
      type: tosca.nodes.Compute
      scalable:
        properties:
          count: 2
      capabilities:
        endpoint:
          properties:
            network_name: PUBLIC
        host:
          properties:
            num_cpus: 2
            mem_size: 2 GB
            disk_size: 40 GB
        os:
          properties:
            type: linux
            distribution: ubuntu
            version: 22.04

  outputs:
    node_ip:
      value: { get_attribute: [ simple_node, public_address, 0 ] }
    node_creds:
      value: { get_attribute: [ simple_node, endpoint, credential, 0 ] }

Setting VMI URI

As in RADL you can set an specific URI identifying the VMI to use in the VM. The URI format is the same used in RADL (System Features). In this case the type must be changed to tosca.nodes.indigo.Compute (the Compute normative type does not support the os image property), and the image property must be added in the os capability:

...

    simple_node:
      type: tosca.nodes.indigo.Compute
      capabilities:
        endpoint:
          properties:
            network_name: PUBLIC
        host:
          properties:
            num_cpus: 2
            mem_size: 2 GB
        os:
          properties:
            image: one://someserver.com/123

...

Advanced Compute host properties

The tosca.nodes.indigo.Compute custom type add a new set of advanced features to the host properties, enabling the request of GPUs and Intel SGX CPU support in the compute node:

...

simple_node:
  type: tosca.nodes.indigo.Compute
  capabilities:
    host:
      properties:
        num_cpus: 2
        mem_size: 2 GB
        num_gpus: 1
        gpu_vendor: nvidia
        gpu_model: Tesla V100
        sgx: false

...

Network properties

Basic properties

The easiest way to specify network requirements of the Compute node is sing the endpoint capability properties. For example the following example the compute node requests for a public IP:

...
    simple_node:
      type: tosca.nodes.Compute
      capabilities:
        endpoint:
          properties:
            network_name: PUBLIC
...

Possible values of the network_name endpoint property:

• PRIVATE: The Compute node does not requires a public IP. This is the default behavior if no endpoint capability is defined.

• PUBLIC: The Compute node requires a public IP.

• Network provider ID: As the provider_id network property in RADL It defines the name of the network in a specific Cloud provider (see _radl_network):

Furthermore the endpoint capability has a set of additional properties to set the DNS name of the node or the set of ports to be externally accesible:

...

  capabilities:
    endpoint:
      properties:
        dns_name: slurmserver
        network_name: PUBLIC
        ports:
          http_port:
            protocol: tcp
            source: 80
            remote_cidr: 0.0.0.0/0 # optional

...

Advanced properties

In case that you need a more detailed definition of the networks, you can use the tosca.nodes.network.Network and tosca.nodes.network.Port TOSCA normative types. In this way you can define the set of networks needed in your topology using the ports to link the networks with the Compute nodes:

...

pub_network:
  type: tosca.nodes.network.Network
  properties:
    network_type: public

server_pub_port:
  type: tosca.nodes.network.Port
  properties:
    order: 1
    ip_address: X.X.X.X # optional to request specific IP
  requirements:
    - binding: simple_node
    - link: pub_network

priv_network:
  type: tosca.nodes.network.Network
  properties:
    network_type: private

server_port:
  type: tosca.nodes.network.Port
  properties:
    order: 0
  requirements:
    - binding: simple_node
    - link: priv_network

...

Custom defined Port type tosca.nodes.indigo.network.Port has a set of additional properties:

• dns_name: DNS name to assing to the network interface.

• additional_ip: (OpenStack specific) Additional IP to be allowed to the network interface.

• additional_dns_names: Additional DNS names.

Software Components

IM enable to use Ansible playbooks as implementation scripts. Furthermore it enables to specify Ansible roles (tosca.artifacts.AnsibleGalaxy.role) and collections (tosca.artifacts.AnsibleGalaxy.collections) to be installed and used in the playbooks:

...

software:
  type: tosca.nodes.SoftwareComponent
  artifacts:
    docker_role:
      file: grycap.docker
      type: tosca.artifacts.AnsibleGalaxy.role
  requirements:
    - host: simple_node
  interfaces:
    Standard:
      configure:
        implementation: https://raw.githubusercontent.com/grycap/ec3/tosca/tosca/artifacts/dummy.yml
        inputs:
          some_input: { get_input: some_input }

...

Storage

IM enables the definition of BlockStorage volumes to be attached to the compute nodes. In this example we can see how to define a volume of 10GB to be attached to the compute node and mounted in the path /mnt/disk. The device parameter is optional and it is only needed in some cloud providers, in general is better not to add it:

...

simple_node:
  type: tosca.nodes.Compute

  ...

  requirements:
    - local_storage:
        node: my_storage
        relationship:
          type: AttachesTo
          properties:
            location: /mnt/disk
            device: hdb # optional

my_storage:
  type: tosca.nodes.BlockStorage
  properties:
    size: 10GB

...

Policies & groups

IM enables the definition of the specific cloud provider where the Compute nodes will be deployed in an hybrid deployment. For example, in the following code we assume that we have defined three computes nodes (compute_one, compute_two and compute_three). We can create a placement group with two of them (compute_one and compute_two) and then set a placement policy with a cloud_id (that must be defined in the Authorization File), and create a second placement policy where we can set a different cloud provider and, optionally, an availability zone:

...

groups:
  my_placement_group:
    type: tosca.groups.Root
    members: [ compute_one, compute_two ]

policies:
  - deploy_group_on_cloudid:
    type: tosca.policies.Placement
    properties: { cloud_id: cloudid1 }
    targets: [ my_placement_group ]

  - deploy_on_cloudid:
    type: tosca.policies.Placement
    properties: { cloud_id: cloudid2, availability_zone: some_zone }
    targets: [ compute_three ]

...

Container Applications (Kubernetes connector)

IM also enables the definition of container applications to be deployed in a Kubernetes cluster. In the following example we can see how to define a container application (IM) that uses a ConfigMap for a configuration file. The IM application is connected with a MySQL backend using the IM_DATA_DB environment variable. The MySQL container is defined with a Persistent Volume Claim (PVC) of 10GB. Furthermore the IM application specifies an endpoint to be published that will result in the creation of a Kubernetes Ingress.

…

node_templates:

im_container:

type: tosca.nodes.Container.Application.Docker properties:

environment:

IM_DATA_DB:

concat:

• “mysql://root:”

• { get_input: mysql_root_password }

• “@”

• { get_attribute: [ mysql_container, endpoints, 0 ] }

• “/im-db”

requirements:

• host: im_runtime

artifacts:

my_image:

file: grycap/im type: tosca.artifacts.Deployment.Image.Container.Docker

my_config_map:

deploy_path: /etc/im/im.cfg file: https://raw.githubusercontent.com/grycap/im/master/etc/im.cfg type: tosca.artifacts.File properties:

# when the content is not provided, the file is downloaded from the URL # otherwise, the file is ignored # If the content is base64 encoded, it is assumed to be a K8s Secret content: |

[im] REST_API = True

# The properties of the runtime to host the container im_runtime:

type: tosca.nodes.Container.Runtime.Docker capabilities:

host:

properties:

num_cpus: 0.5 mem_size: 1 GB publish_ports:

• protocol: tcp target: 8800 source: 30880 endpoint: https://im.domain.com/im

# The MYSQL container based on official MySQL image in Docker hub mysql_container:

type: tosca.nodes.Container.Application.Docker properties:

environment:

MYSQL_ROOT_PASSWORD: { get_input: mysql_root_password } MYSQL_DATABASE: “im-db”

requirements:

• host: mysql_runtime

artifacts:

my_image:

file: mysql:8 type: tosca.artifacts.Deployment.Image.Container.Docker

# The properties of the runtime to host the container mysql_runtime:

type: tosca.nodes.Container.Runtime.Docker capabilities:

host:

properties:

num_cpus: 0.5 mem_size: 1 GB expose_ports:

• protocol: tcp target: 3306

volumes:

• “some_vol:/var/lib/mysql”

some_vol:

type: tosca.nodes.BlockStorage properties:

size: 10 GB # Set the PV name in this field # volume_id: “PV name”

outputs:

im_service_endpoint:

value: { get_attribute: [ im_container, endpoints, 0 ] }

Advanced Output values

The tosca.nodes.indigo.Compute node type adds a new attribute named: ansible_output. It is a map that has one element per each IM configuration step, so you can access it by name. The steps have the keyword tasks that is also a map that has one element per ansible task. In this case it can bes accessed using the task name as defined in the playbook. Finally there is an output keyword that returns the output of the task. In most of the cases the task is a debug ansible task that shows anything you want to return.

In the following example the specified task was a debug ansible task that shows the value of a internal defined value:

...

  outputs:
    node_ip:
      value: { get_attribute: [ front, ansible_output, lrms_front_end_front_conf_front, tasks, 'grycap.nomad : nomad_secret_id', output ] }