== Verification of cesnet.cloudkeeper.centos7.x86_64-1.6.0 and cesnet.cloudkeeper-one.centos7.x86_64-1.3.0 == === Ticket assigned === * [https://rt.egi.eu/rt/Ticket/Display.html?id=13965] * [https://rt.egi.eu/rt/Ticket/Display.html?id=13966] === CloudKeeper repo and instalation === For dependence: {{{ [root@verification ~]# yum install -y centos-release-qemu-ev [root@verification ~]# wget http://repository.egi.eu/sw/production/cas/1/current/repo-files/EGI-trustanchors.repo -O /etc/yum.repos.d/EGI-trustanchors.repo [root@verification ~]# wget http://lcg-ca.web.cern.ch/lcg-ca/distribution/current/repo-files/lcg-trustanchors.repo -O /etc/yum.repos.d/lcg-trustanchors.repo [root@verification ~]# yum install -y lcg-CA }}} {{{ [root@verification ~]# wget http://admin-repo.egi.eu/sw/unverified/cmd-one-1.cesnet.cloudkeeper.centos7.x86_64/1/6/0/repofiles/CESNET.cloudkeeper.centos7.x86_64.repo -O /etc/yum.repos.d/CESNET.cloudkeeper.centos7.x86_64.repo [root@verification ~]# wget http://admin-repo.egi.eu/sw/unverified/cmd-one-1.cesnet.cloudkeeper-one.centos7.x86_64/1/3/0/repofiles/CESNET.cloudkeeper-one.centos7.x86_64.repo -O /etc/yum.repos.d/CESNET.cloudkeeper-one.centos7.x86_64.repo [root@verification ~]# cat /etc/yum.repos.d/CESNET.cloudkeeper.centos7.x86_64.repo # EGI Software Repository - REPO META (releaseId,repositoryId,repofileId) - (13965,2538,2498) [CESNET.cloudkeeper.centos7.x86_64] name=CESNET.cloudkeeper.centos7.x86_64 baseurl=http://admin-repo.egi.eu/sw/unverified/cmd-one-1.cesnet.cloudkeeper.centos7.x86_64/1/6/0/ enabled=1 protect=1 priority=1 gpgcheck=1 gpgkey=http://repository.egi.eu/sw/production/umd/UMD-RPM-PGP-KEY }}} {{{ [root@verification ~]# cat /etc/yum.repos.d/CESNET.cloudkeeper-one.centos7.x86_64.repo # EGI Software Repository - REPO META (releaseId,repositoryId,repofileId) - (13966,2539,2499) [CESNET.cloudkeeper-one.centos7.x86_64] name=CESNET.cloudkeeper-one.centos7.x86_64 baseurl=http://admin-repo.egi.eu/sw/unverified/cmd-one-1.cesnet.cloudkeeper-one.centos7.x86_64/1/3/0/ enabled=1 protect=1 priority=1 gpgcheck=1 gpgkey=http://repository.egi.eu/sw/production/umd/UMD-RPM-PGP-KEY }}} {{{ [root@verification ~]# LC_ALL=C yum install cloudkeeper cloudkeeper-one [.....] Dependencies Resolved ============================================================================================================================================================================================= Package Arch Version Repository Size ============================================================================================================================================================================================= Installing: cloudkeeper x86_64 1.6.0+20180104132221-3.el7 CESNET.cloudkeeper.centos7.x86_64 41 M cloudkeeper-one x86_64 1.3.0+20180105125148-3.el7 CESNET.cloudkeeper-one.centos7.x86_64 52 M Installing for dependencies: boost-system x86_64 1.53.0-27.el7 base 40 k boost-thread x86_64 1.53.0-27.el7 base 57 k glusterfs x86_64 3.8.4-53.el7.centos base 529 k glusterfs-api x86_64 3.8.4-53.el7.centos base 75 k glusterfs-client-xlators x86_64 3.8.4-53.el7.centos base 789 k glusterfs-libs x86_64 3.8.4-53.el7.centos base 370 k gperftools-libs x86_64 2.6.1-1.el7 base 272 k libaio x86_64 0.3.109-13.el7 base 24 k libiscsi x86_64 1.9.0-7.el7 base 60 k librados2 x86_64 1:0.94.5-2.el7 base 1.7 M librbd1 x86_64 1:0.94.5-2.el7 base 1.8 M qemu-img-ev x86_64 10:2.10.0-21.el7_5.3.1 centos-qemu-ev 1.2 M Transaction Summary ============================================================================================================================================================================================= Install 2 Packages (+12 Dependent packages) Total download size: 99 M Installed size: 252 M Is this ok [y/d/N]: y [.....] Retrieving key from http://repository.egi.eu/sw/production/umd/UMD-RPM-PGP-KEY Importing GPG key 0x96B71B07: Userid : "Kostas Koumantaros (UMD Release Manager) " Fingerprint: 32ad 8d80 fa5a 89b5 3dc5 de93 6799 de16 96b7 1b07 From : http://repository.egi.eu/sw/production/umd/UMD-RPM-PGP-KEY Is this ok [y/N]: y Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Virtualization Importing GPG key 0x61E8806C: Userid : "CentOS Virtualization SIG (http://wiki.centos.org/SpecialInterestGroup/Virtualization) " Fingerprint: a7c8 e761 309d 2f1c 92c5 0b62 7aeb be82 61e8 806c Package : centos-release-virt-common-1-1.el7.centos.noarch (@extras) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Virtualization Is this ok [y/N]: y [.....] Installed: cloudkeeper.x86_64 0:1.6.0+20180104132221-3.el7 cloudkeeper-one.x86_64 0:1.3.0+20180105125148-3.el7 Dependency Installed: boost-system.x86_64 0:1.53.0-27.el7 boost-thread.x86_64 0:1.53.0-27.el7 glusterfs.x86_64 0:3.8.4-53.el7.centos glusterfs-api.x86_64 0:3.8.4-53.el7.centos glusterfs-client-xlators.x86_64 0:3.8.4-53.el7.centos glusterfs-libs.x86_64 0:3.8.4-53.el7.centos gperftools-libs.x86_64 0:2.6.1-1.el7 libaio.x86_64 0:0.3.109-13.el7 libiscsi.x86_64 0:1.9.0-7.el7 librados2.x86_64 1:0.94.5-2.el7 librbd1.x86_64 1:0.94.5-2.el7 qemu-img-ev.x86_64 10:2.10.0-21.el7_5.3.1 Complete! }}} === CloudKeeper and CloudKeeper-one configuration === {{{ [root@verification ~]# mkdir -p /etc/grid-security/vomsdir/ops/ [root@verification ~]# cat /etc/grid-security/vomsdir/ops/lcgvoms24.cern.ch.lsc /DC=ch/DC=cern/OU=computers/CN=lcg-voms2.cern.ch /DC=ch/DC=cern/CN=CERN Grid Certification Authority [root@verification ~]# cat /etc/grid-security/vomsdir/ops/voms2.cern.ch.lsc /DC=ch/DC=cern/OU=computers/CN=voms2.cern.ch /DC=ch/DC=cern/CN=CERN Grid Certification Authority }}} {{{ [root@verification ~]# chmod +w /etc/cloudkeeper/cloudkeeper.yml }}} {{{ [root@verification ~]# cat /etc/cloudkeeper/cloudkeeper.yml cloudkeeper: image-lists: # List of image lists to sync against - https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:x-oauth-basic@vmcaster.appdb.egi.eu/store/vo/ops/image.list image-lists-file: # File containing list of image lists to sync against ca-dir: /etc/grid-security/certificates/ # CA directory authentication: false # core (client) <-> backend (server) authentication (certificate, key and backend-certificate options) certificate: /etc/grid-security/hostcert.pem # Core's host certificate key: /etc/grid-security/hostkey.pem # Core's host key image-dir: /var/spool/cloudkeeper/images/ # Directory to store images to external-tools: binaries: qemu-img: /usr/bin/qemu-img # qemu-img binary (image conversion) location nginx: /opt/cloudkeeper/embedded/sbin/nginx # nginx binary (HTTP server) location execution-timeout: 600 # timeout for execution of external tools in seconds remote-mode: true # Remote mode starts HTTP server (NGINX) and serves images to backend via HTTP nginx: runtime-dir: /var/run/cloudkeeper/ # Runtime directory for NGINX error-log-file: /var/log/cloudkeeper/nginx-error.log # File for NGINX error log access-log-file: /var/log/cloudkeeper/nginx-access.log # File for NGINX access log pid-file: /var/run/cloudkeeper/nginx.pid # NGINX pid file ip-address: 193.144.35.111 # IP address NGINX can listen on (the IP in which cloudkeeper is installed) port: 50505 # Port NGINX can listen on proxy: ip-address: # Proxy IP address port: # Proxy port ssl: false # Whether proxy will use SSL connection backend: endpoint: 127.0.0.1:50051 # Backend's gRPC endpoint certificate: /etc/grid-security/backendcert.pem # Backend's certificate formats: # List of acceptable formats images can be converted to - qcow2 logging: level: ERROR # Logging level file: /var/log/cloudkeeper/cloudkeeper.log # File to write log to. To turn off file logging leave this field empty. lock-file: /var/lock/cloudkeeper/cloudkeeper.lock # File used to ensure only one running instance of cloudkeeper debug: false # Debug mode }}} {{{ [root@verification ~]# cat /etc/cloudkeeper-one/cloudkeeper-one.yml cloudkeeper-one: listen-address: 127.0.0.1:50051 # IP address gRPC server will listen on authentication: false # core (client) <-> backend (server) authentication (certificate, key and core-certificate options) certificate: /etc/grid-security/hostcert.pem # Backend's host certificate key: /etc/grid-security/hostkey.pem # Backend's host key identifier: cloudkeeper-one # Instance identifier core: certificate: /etc/grid-security/corecert.pem # Core's certificate appliances: tmp-dir: /var/spool/cloudkeeper-one/appliances # Directory where to temporarily store appliances template-dir: /etc/cloudkeeper-one/templates/ # If set, templates within this directory are used to construct images and templates in OpenNebula permissions: "640" # UNIX-like permissions appliances will have within OpenNebula opennebula: secret: oneadmin:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx # If not specified, looking for secret in environment variable ONE_AUTH and file ~/.one/one_auth endpoint: http://193.146.75.190:2633/RPC2 # If not specified, looking for endpoint in environment variable ONE_XMLRPC and file ~/.one/one_endpoint datastores: # Array of OpenNebula datastores images will be uploaded to - default users: # Handle only images/templates of specified users api-call-timeout: 3h # How long will cloudkeeper-one wait for image/template operations to finish in OpenNebula allow-remote-source: true # Allows OpenNebula to directly download remote image logging: level: ERROR # Logging level file: /var/log/cloudkeeper-one/cloudkeeper-one.log # File to write log to. To turn off file logging leave this field empty. debug: false # Debug mode }}} {{{ [root@verification ~]# cat /etc/cloudkeeper-one/templates/template.erb NAME = "<%= name %>" <% if appliance.description %> DESCRIPTION = "<%= "#{appliance.title} - #{appliance.description}" %>" <% else %> DESCRIPTION = "<%= appliance.title %>" <% end %> MEMORY = "<%= appliance.ram != 0 ? appliance.ram : 1024 %>" CPU = "<%= appliance.core != 0 ? appliance.core : 0.25 %>" VCPU = "<%= appliance.core != 0 ? appliance.core : 1 %>" OS = [ ARCH = "<%= appliance.architecture || "x86_64" %>" ] DISK = [ IMAGE_ID = "<%= image_id %>" ] NIC=[ NETWORK="fedcloud-main", NETWORK_UNAME="oneadmin" ] GRAPHICS=[ KEYMAP="es", LISTEN="0.0.0.0", TYPE="VNC" ] CONTEXT = [ NETWORK = "YES", SSH_PUBLIC_KEY = "$USER[SSH_PUBLIC_KEY]" ] }}} (!) Some fix for permission / ownership / directory creation was necessary: {{{ [root@verification ~]# chown cloudkeeper-one. /var/log/cloudkeeper-one [root@verification ~]# mkdir /var/log/cloudkeeper [root@verification ~]# chown cloudkeeper. /var/log/cloudkeeper [root@verification ~]# mkdir /var/lock/cloudkeeper [root@verification ~]# chown cloudkeeper. /var/lock/cloudkeeper [root@verification ~]# mkdir /run/cloudkeeper [root@verification ~]# chown cloudkeeper. /run/cloudkeeper }}} (!) The following ones was due to the need of edit these files for configuration purposes: {{{ [root@verification ~]# chmod +w /etc/cloudkeeper/cloudkeeper.yml [root@verification ~]# chmod +w /etc/cloudkeeper-one/cloudkeeper-one.yml [root@verification ~]# chmod +w /etc/cloudkeeper-one/templates/template.erb }}} === CloudKeeper run and test === {{{ [root@verification ~]# systemctl enable cloudkeeper-one; systemctl start cloudkeeper-one Created symlink from /etc/systemd/system/multi-user.target.wants/cloudkeeper-one.service to /etc/systemd/system/cloudkeeper-one.service. }}} (!) Disable firewall both in one box and this machine And run the command: {{{ [root@verification ~]# sudo -u cloudkeeper /opt/cloudkeeper/bin/cloudkeeper --logging-level=DEBUG 2018-05-31T09:24:34-04:00 [DEBUG] 3102 : Running in debug mode... 2018-05-31T09:24:34-04:00 [DEBUG] 3102 : Cloudkeeper 'sync' called with parameters: {"logging-level"=>"DEBUG", "logging-file"=>"/var/log/cloudkeeper/cloudkeeper.log", "lock-file"=>"/var/lock/cloudkeeper/cloudkeeper.lock", "debug"=>false, "image-lists"=>["https://bbde69ff-3bd5-45ac-8199-523d26bdcf5e:x-oauth-basic@vmcaster.appdb.egi.eu/store/vo/ops/image.list"], "ca-dir"=>"/etc/grid-security/certificates/", "authentication"=>false, "certificate"=>"/etc/grid-security/hostcert.pem", "key"=>"/etc/grid-security/hostkey.pem", "image-dir"=>"/var/spool/cloudkeeper/images/", "qemu-img-binary"=>"/usr/bin/qemu-img", "nginx-binary"=>"/opt/cloudkeeper/embedded/sbin/nginx", "external-tools-execution-timeout"=>600, "remote-mode"=>true, "nginx-runtime-dir"=>"/var/run/cloudkeeper/", "nginx-error-log-file"=>"/var/log/cloudkeeper/nginx-error.log", "nginx-access-log-file"=>"/var/log/cloudkeeper/nginx-access.log", "nginx-pid-file"=>"/var/run/cloudkeeper/nginx.pid", "nginx-ip-address"=>"193.144.35.111", "nginx-port"=>50505, "nginx-proxy-ssl"=>false, "backend-endpoint"=>"127.0.0.1:50051", "backend-certificate"=>"/etc/grid-security/backendcert.pem", "formats"=>["qcow2"]} 2018-05-31T09:24:35-04:00 [DEBUG] 3102 : Running appliance synchronization... 2018-05-31T09:24:35-04:00 [DEBUG] 3102 : 'pre_action' gRPC method call 2018-05-31T09:24:35-04:00 [DEBUG] 3102 : 'image_lists' gRPC method call 2018-05-31T09:24:35-04:00 [DEBUG] 3102 : Downloading fresh image lists... 2018-05-31T09:24:35-04:00 [DEBUG] 3102 : Downloading image list from "https://bbde69ff-3bd5-45ac-8199-523d26bdcf5e:x-oauth-basic@vmcaster.appdb.egi.eu/store/vo/ops/image.list" 2018-05-31T09:24:35-04:00 [DEBUG] 3102 : Removing appliances from expired image lists... 2018-05-31T09:24:35-04:00 [DEBUG] 3102 : Registering appliances from new image lists... 2018-05-31T09:24:35-04:00 [DEBUG] 3102 : Image lists to register: ["fc5fc591-2fa2-59ff-9f35-47a63a111d4f"] 2018-05-31T09:24:36-04:00 [DEBUG] 3102 : Downloading image from "https://cephrgw01.ifca.es:8080/swift/v1/egi_endorsed_vas/Small.Ubuntu.16.04-2018.03.12.ova" 2018-05-31T09:24:48-04:00 [DEBUG] 3102 : Executing command: ["file", "-b", "/var/spool/cloudkeeper/images/Small.Ubuntu.16.04-2018.03.12.ova"] 2018-05-31T09:24:48-04:00 [DEBUG] 3102 : Executing command: ["tar", "-t", "-f", "/var/spool/cloudkeeper/images/Small.Ubuntu.16.04-2018.03.12.ova"] 2018-05-31T09:24:51-04:00 [DEBUG] 3102 : Converting file "/var/spool/cloudkeeper/images/Small.Ubuntu.16.04-2018.03.12.ova" from :ova to "qcow2" 2018-05-31T09:24:51-04:00 [DEBUG] 3102 : Converting file "/var/spool/cloudkeeper/images/Small.Ubuntu.16.04-2018.03.12.ova" from :ova to vmdk 2018-05-31T09:24:51-04:00 [DEBUG] 3102 : Executing command: ["tar", "-t", "-f", "/var/spool/cloudkeeper/images/Small.Ubuntu.16.04-2018.03.12.ova"] 2018-05-31T09:24:51-04:00 [DEBUG] 3102 : Executing command: ["tar", "-x", "-f", "/var/spool/cloudkeeper/images/Small.Ubuntu.16.04-2018.03.12.ova", "-C", "/var/spool/cloudkeeper/images", "Small.Ubuntu.16.04-2018.03.12-disk001.vmdk"] 2018-05-31T09:25:09-04:00 [DEBUG] 3102 : Converting file "/var/spool/cloudkeeper/images/Small.Ubuntu.16.04-2018.03.12-disk001.vmdk" from :vmdk to "qcow2" 2018-05-31T09:25:09-04:00 [DEBUG] 3102 : Executing command: ["/usr/bin/qemu-img", "convert", "-f", "vmdk", "-O", "qcow2", "/var/spool/cloudkeeper/images/Small.Ubuntu.16.04-2018.03.12-disk001.vmdk", "/var/spool/cloudkeeper/images/Small.Ubuntu.16.04-2018.03.12-disk001.qcow2"] 2018-05-31T09:26:08-04:00 [DEBUG] 3102 : 'add_appliance' gRPC method call (appliance.identifier: e29454eb-d751-5bf8-9f81-91f5a4aac7ec) 2018-05-31T09:26:08-04:00 [DEBUG] 3102 : Starting NGINX server 2018-05-31T09:26:08-04:00 [DEBUG] 3102 : Prepared NGINX authentication file "/tmp/cloudkeeper-nginx-auth20180531-3102-bya290": username: "d02c3e25-f55c-4a8f-9a3a-1574282907d7", password: "a33f8998-b314-4c74-9065-019fbbd991d7" 2018-05-31T09:26:08-04:00 [DEBUG] 3102 : NGINX configuration: {:error_log_file=>"/var/log/cloudkeeper/nginx-error.log", :access_log_file=>"/var/log/cloudkeeper/nginx-access.log", :pid_file=>"/var/run/cloudkeeper/nginx.pid", :auth_file=>"/tmp/cloudkeeper-nginx-auth20180531-3102-bya290", :root_dir=>"/var/spool/cloudkeeper/images", :image_file=>"Small.Ubuntu.16.04-2018.03.12-disk001.qcow2", :ip_address=>"193.144.35.111", :port=>50505, :proxy_ip_address=>nil, :proxy_port=>nil, :proxy_ssl=>false} 2018-05-31T09:26:08-04:00 [DEBUG] 3102 : Prepared NGINX configuration file "/tmp/cloudkeeper-nginx-conf20180531-3102-w9cjd8": worker_processes 1; error_log /var/log/cloudkeeper/nginx-error.log; pid /var/run/cloudkeeper/nginx.pid; events { worker_connections 1024; } http { default_type application/octet-stream; access_log /var/log/cloudkeeper/nginx-access.log; sendfile on; sendfile_max_chunk 1m; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; auth_basic "Cloudkeeper image"; auth_basic_user_file /tmp/cloudkeeper-nginx-auth20180531-3102-bya290; server { listen 50505; server_name 193.144.35.111; root /var/spool/cloudkeeper/images; location / { try_files /Small.Ubuntu.16.04-2018.03.12-disk001.qcow2 /Small.Ubuntu.16.04-2018.03.12-disk001.qcow2; } } } 2018-05-31T09:26:08-04:00 [DEBUG] 3102 : Executing command: ["/opt/cloudkeeper/embedded/sbin/nginx", "-c", "/tmp/cloudkeeper-nginx-conf20180531-3102-w9cjd8", "-p", "/var/run/cloudkeeper/"] 2018-05-31T09:26:28-04:00 [DEBUG] 3102 : Stopping NGINX server 2018-05-31T09:26:28-04:00 [DEBUG] 3102 : Executing command: ["/opt/cloudkeeper/embedded/sbin/nginx", "-s", "stop", "-c", "/tmp/cloudkeeper-nginx-conf20180531-3102-w9cjd8", "-p", "/var/run/cloudkeeper/"] 2018-05-31T09:26:28-04:00 [DEBUG] 3102 : Cleaning downloaded image files for appliance "e29454eb-d751-5bf8-9f81-91f5a4aac7ec" 2018-05-31T09:26:29-04:00 [DEBUG] 3102 : Synchronizing registered appliances... 2018-05-31T09:26:29-04:00 [DEBUG] 3102 : Image lists to synchronize: [] 2018-05-31T09:26:29-04:00 [DEBUG] 3102 : 'post_action' gRPC method call }}} After finish without errors, images and templates are syncronized in OpenNebula... === Finding world-writable files in the packages contents === {{{ [root@verification ~]# rpm -qalv | egrep "^[-d]([-r][-w][-xs]){2}[-r]w" drwxrwxrwt 2 root root 0 abr 11 00:59 /tmp drwxrwxrwt 2 root root 0 abr 11 00:59 /var/tmp }}}