Terra and security researcher and industry. I will show how to secrets, keys in Docker containers and kubernetes in different environments and also comment best practice for saving and securing distribution of secrets using specific tools in the devsecops ecosystem. BSDR have written public is this book with Italy implemented Docker and Kubernetes. In this book, basically you will learn about Docker and Kubernetes as a platform for the development of containers taking into account the security of applications. Also you will learn for example how to secure your Docker and Kubernetes environments and discover vulnerabilities and threads with specific tools like Falco, three eco events and so on. These could be the main points of the tools. First, I will commend the challenge of security and secret keys in containers ladder. I will commend the best practice for securing a security distributing of secrets in Docker containers. For example, with Docker secrets we can manage information that is needed at real time but cannot be exposed in the Dockerimas or source code repository. We continue with managing secrets in Kubernetes using volumes and sales secrets and finally I will comment other tools for distributing secrets in containers like Hashicorp but and keyword we start commenting the tools factor the tools factor is a methodology that allows to create software service applications that are deployed that are usually deployed in encloded environments. These methodology includes best practice for applications are developed with maximum portability, stability, security and fault tolerance. One of the many philosophy of twelve factor apps is that configuration should be stored in the environment. In practical terms, these means storing configuration separating from other source code. In general, secrets provide a mechanism to store data that can be read by application at runtime and secrets play a critical role in security when storing sensitive data separately from application code. This include data such as passwords, hostname, key certificates and so on. For example, if your application requires a database connection to this, usually you need a hostname, a username, a password for connecting these database. Furthermore, there's a different database server for development, testing, production and so on. And in this way secrets with secrets you can meet that each environment can provide its own database information to the applications and the application code doesn't needed to know which environment it is securing. Basically, a credential manager or secrets manager basically is an administrator of credential passwords, configurations or certificates. The main reason why our applications are going to need a secret manager is mainly to prevent the most sensitive information from being leaked. In this way we can compare a secrets manager to a user password manager. These a secrets manager will work at the application level, sharing credential and connect and connecting certificates in this way. For example, Docker secrets is a treaty that can help us to store any type of data that can be represented as a string or a binary. At this point, we can include username and passwords, hostname and port sh keys, TLA certificates, and so on. The following diagram shows how Docker use RAF that is a consensus algorithm for managing a replicated log in a distributed system to ensure that data remains consistent in the NT swart and all the service running on it. Basically, the RAF algorithm transmits a secret in an incretive way to all the manager nodes. Other diagrams also shows the worker nodes which are instances of Docker engine securing these containers. At this point, Docker secrets works like a bolt where you can pull sensitive data and you can only use it if you have the bolt keys. You can only access this data if you have the master key or the bulk keys, which you must get from the service code to which you actually need other key. Currently, Docker secrets are only available to swar service. This means that standalone containers cannot access Siggrass. Therefore, to use Sigress, we must configure our cluster for swar for Docker swar using the Docker swar init command where the manager IP is the IP address accented by Docker to the managing node. And when you add a secret to the swar, Docker sends the secret to the swarmanager over a mutual delays connection. For example, we could use the contents of a file to create a secret and when you grant a newly created or running service access to a secret, the secret is mounted into the container in can in memory file system. The location of the mount point within the containers by default is secrets name. Due to Docker secrets require our cluster to be in Swarmode, we cannot access other secrets from the normal Docker run command. Instead, we have to create services and we can specify when creating services. At this moment we can create secrets associated to these services. In this example we are creating ready services and granting access to other secret that we have created. Next, we can for example get the identifier of the ready service tax container using the Docker PS command and at this point you can use Docker containers, a command to connect to the container and read the context of the secret data file. When you add or inspect an individual secret at any time or list all secrets, the limitation is that you cannot remove a secret that is running services is using. If you try removing the secret, the removal file because other Reddit service is running and has access to the secret. In this example, we are removing the secret by the service and to remove normally what is we want to remove a secret without distributing the running service. You can use other techniques like rotating secrets using other techniques. For example, establishing a policy of your secret keys. Next, for example, we call verify other service no longer has access to the secrets. In this way a node only has access to encrypt secrets and if the code only has access to a secret, if other node is a swarm manager or if this is running service tasks which have been rated access to the secret and when a container tasks stop running, the decrypted secrets are unmounted from the memory system file system for that container and fluid from the code memory. We continue commenting. Best practice associated with when we needed to secure this kind of information. For example, is it considered a best practice to remove a secret once it's no longer needed? For example, we can permanently remove a secret using the command Docker secret rein secrets. At this point it's important to remember that before remove the secret, we need to stop and remove services that are using this secret docker. Another best practice for example is relative with Docker that makes secrets available to our application as files. As we have seen before, the could behavior is to make each secret its own file in the directory run secrets. But this default behavior can be changed if we change the location of the file by securing it with our service. For example, the following command is useful is the secret if the secret contains information that our application expects in a specific location. The secret functionalities are available using Docker compose the following file configuration we define two secrets other first one is external, meaning it refers to a secret created using the docker secret command and the second one refers to a file and doesn't require any initial setup with Docker. Keep in mind that using the file approach bypass most of other benefits of using Docker secret. Also, other file must be available to all hosts where the service might run my execute and we must take care to protect the content of the file using mechanisms other than Docker. For example, using encryption techniques over this file that we are sharing in the docker host. For example, docker composite tasks can be deployed using other docker client and as part of the developed deployment other stack will be configured with access to the secret and the following commas allow deploying the services and access other secrets that we are deploying with Docker stack. Now we are commenting the same button in Kubernetes when working with Kubernetes well, the most easy way to create a secret is using the Kubectl command to reference the manifest file you just created and the request will be sent to the APA server in the Kubernetes control plane for the request to be action. Afterward, other data will be stored in the ETC data store of your cluster when worker go in. Kubernetes secrets also can also be passed to containers using mounted volumes. This will cause the configuration data to appear in files available to the container file system and each top level k in the configuration data will appear as a file containing all case below that top level k. Within the Kubernetes ecosystem we can find other projects that allow the management of secrets. For example, Seal secrets project is a tool that allows to encrypt secrets using a resource called seed secrets. This tool can be used to encrypt Kubernetes secrets and store it on the GitHub repository. In this way seed secret encrypts the secret data and it can be decrypt using the Kubernetes crusted controller. The main components of these solutions are the client used to generate encrypted secrets. At this point the user don't have the capacity to decrypt except the secret controller and the other components is other Kubernetes secret controller used to decrypt the seal secret and store it as a secret in the Kubernetes cluster. The solution is based on a publicly infrastructure and serves a public key to encrypt and install a private key on the cluster. Basically this tool encrypts data using a public key for the cluster using the certificate of the Kubernetes cluster when it's applied in the following configuration file we are defining a secret that containers user key and password in base 64 format which are easily reversible for securing this configuration file that containers critical information. You can other create a secret and seal it with Kubernetes solute that can be found in the GitHub repository of the project and the following command generates encrypt data for the secret and is not possible to decrypt except using the sealed secret controller. This is the encrypt data for the secret generated with the Kubernetes Seal command and at this point no one can decrypt except the CFC controller. The previous manifest file for example can be used. We can use the previous manifest file to create the seal secret in your cluster and can be executed using the Kubectl apply command and at this point this seal secret can be used to be pushed to the jeep repository without the consent of any individual accessing other original volumes. Docker and Kubernetes secrets are SLS solutions, but you can also consider third party secret restoration solutions. Some popular secrets management alternatives include Hashicorp, Kweeth, Achilles, Bolt. Also we have cloud provider solutions, specific solutions for AWS, Google Cloud, Google Cloud and so on, Azure and so on. For example, we can start with a tool for storing and accessing secrets in a securing way that offers an interface for storing secrets while providing an access control system and an access lock system. In these diagram we highlight how Bolt make a separation between the different components. On one hand we have the storage component and on the other hand we have the components to generate the secrets and credentials to access these secrets. When bolt is start for the fish time, an encryption key generated and is used to protect all information. This key is protected by a master key which by default is split into five shaded keys and at these point we will be able to generate the master keys with three of these five shaded keys and with this case we could access the secrets. Secrets are encrypted outside bowel using what is called Bowel's transit secrets giant is can encryption as a service solution to be secure? Bowl performs cryptographic funtios on data intransit and doesn't store data while the data is intransit. Bow, like other hashicorp tools such as console and Nomad, provides a graphical user interface accessible through the roster that allows other same operations to be executed through the API or from the command line. These are the keys features of other valve. We can highlight features like the capacity to encrypt and decrypt data without storing it. Other data have the capacity also to generate secrets on demand for certain operations. Also allows the replication across multiple data centers, offers protection for secret renovation and ships as a secret repository with access control details and another interesting tool is keyword in a similar way that we have seen with all kwith helps with infrastructure tools, secrets, GPG carriers, database credentials, tilly certificates and so on. These are the main components of these solutions are the kwith server that provides JSON API for accessing and managing the secrets KCNC that is a kwith client that retrieves secrets from the keyword server using tools with a client certificates. Other client Other K with CLE is the main tool for managing the secrets for managing the secrets and the users that can now take these secrets and the automotive API is the API we can use for request access to these secrets. For example, in this script capture, we can see where we can add a secret using K with clip and the K with automation API for making a request using a client certificate. The main features of kwith are basically kwith helps with the main advantage of this tool is that has the capacity to store all secrets in memory only. Another interesting solution that provides cloud providers, for example IBls secret managers lets you quickly rotate, manage and retrieve database credentials, APK and passwords. Using these solutions you can secure, analyze and manage secrets needed to access the AWS cloud capabilities on third party service and on premise. This is the main features of these solutions. It has the capacity to encrypt other secrets transmitting over TLS in a way that we have seen with Hashicorp and Kwith also provides client side catching libraries to improve the availability and reduce the latency of using your secret. And obviously you can use a specific service from AWS. For example you can use BPC virtual private club. You can configure this service these close service to keep traffic within the OS network. If you are hosting your application on Azure, for example, there is a tool specific called Azure cable that enables users to manage all secrets for their applications that are deployed in the Azure cloud provider. This tool has the capacity to ensure that all your secrets are managing in a single repository. This includes key certificates, connection strings and password and you can use K Vault's Rest API for connecting your application with the Azure cable. Another tool that we can see in this ecosystem is Aki less but that is similar tool that basically provides sower as a service platform that has the capacity to protect credentials both static dynamic including third certificate, automatician and encryption keys. This platform supports the main features of this tool that provides a serotras application access by providing unify authentication and provides also an encryption as a service allowing customers to protect sensitive data and useless data by applying feeds 142 certificate certified up level encryption. This kind of certificate or certificate is a security standard for accreditation of cryptographic models used in other cryptography industry mainly. And finally for concluding remember that not stored secrets in the application system environment variables or source code managing system and the main recommendation is using secrets as a main tool for any container based architecture application because they help to achieve the goal of keeping code and configurations in asylum separate and recommend adopt some of these managing solutions because sensitive information can be easily leaked and out of the credentials could make applications vulnerable. And that's all. Thank you very much.