AWS / Azure / GCP Cloud | Database | Data Analytics | LakeHouse | Machine Learning | Shrenik Parekh

Docker: Docker is an engine by Docker. Explore slides . Runs on host OS as well as guest OS, bare metal, cloud. Image: Packages software code, run-time, system tools, system libraries and settings. Images become containers when they run on Docker Engine. Container:   Standard unit of software. Isolate software from its environment Software run same on Windows and Linux. Could be portable anywhere. Abstraction at the app layer that packages code and all dependencies together. Runs application quickly from one computing env. to another env. Do not require an OS per application, as it share the machine’s OS system kernel Applications are safer in containers Multiple containers can run on the same machine Each container runs as an isolated processes in user space. Require less space than VM. Virtual Machines: VMs are an abstraction of physical hardware. Turns one server into many servers. The hypervisor allows multiple VMs to run on a single machine. Each V...

Kubernetes directory structure and process on Control Plane / Master: Kubelet environment file - /var/lib/kubelet/kubeadm-flags.env Kubelet configuration file - /var/lib/kubelet/config.yaml Certificate directory folder - /etc/kubernetes/pki Kube config directory - /etc/kubernetes admin config file - /etc/kubernetes/admin.conf kube config file - /etc/kubernetes/kubelet.conf control manager config file - /etc/kubernetes/controller-manager.conf scheduler config file - /etc/kubernetes/scheduler.conf Manifest directory - /etc/kubernetes/manifests Cluster store - /etc/kubernetes/manifests /etc/kubernetes/manifests/etcd.yaml  /etc/kubernetes/manifests/kube-apiserver.yaml  /etc/kubernetes/manifests/kube-controller-manager.yaml  /etc/kubernetes/manifests/kube-scheduler.yaml /etc/kubernetes/pki: Has following key, crt files and one directory for etcd apiserver.crt apiserver.key ca.crt  front-proxy-ca.crt front-proxy-client.key apiserver-etcd-client.crt  apiserver-kub...

Install Kubernetes Control Plane (master) and Worker Node: Pre-requisite for Kubernetes Control Plane and Worker Nodes / minion: Disable selinux: Edit /etc/selinux/config file, and set SELINUX-disabled as follows, and bounce the server. SELINUX=disabled #getenfoce Disabled Disable firewall: #systemctl stop firewalld #systemctl disable firewalld #stystemctl status firewalld Active: inactive (dead) Set bridge-nf-call-iptables contents: echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables Oterwise you will get following error [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1 Disable swap: #swapoff -a Otherwise you will get following error [ERROR Swap]: running with swap on is not supported. Please disable swap Configure Repository on control plane and worker node: Execute following command to create file /etc/repos.d/kubernetes.repo and add content for kubernetes, which will help to set na...

Create MySQL custom image: MySQL custom image can be created using Dockerfile: Create custom .sql script which could have DDL, DML, Meta Data: /mysql/docker_compose1/sql_scripts.sql cd to directory /mysql/docker_compose1/: Create Dockerfile: Include following into the file Dockerfile FROM mysql COPY ./sql_scripts.sql/ /docker-entrypoint-initdb.d/ Create own MySQL image: #docker build -t docker_compose1 .

MySQL Custom Docker Container | Docker Compose: Here we are going to explore how to customize, MySQL Docker container using custom parameters in file, using volume, and Docker compose: Create parameter file my.cnf locally, create Docker file which has mount volume (directory) configuration, bring up container using "docker-compose up -d", connect database and make sure everything is as expected. Create directory for persistent data volume: Create directory /mysql/mysql8020_data/docker_compose1/data_1 to store MySQL docker container persistent data. Create my.cnf file at /mysql/docker_compose_1/conf.d/: [mysqld] default_authentication-plugin=mysql_native_password server_id=7777 port=7399 Note: You might be wondering how local conf.d director's my.cnf file is read by docker container. Local my.cnf file is mapped to container /etc/mysql/conf.d directory which can be access by following command. File my.cnf is not mapped, directory conf.d is mapped to /etc/m...

DC/OS 2.2 Production installation on Linux: Make certain to refer Before Installation 01. Download DCOS V2.2 installation file -dcos_generate_config.sh from https://docs.d2iq.com/mesosphere/dcos/2.0/release-notes/2.0.2/ for Open source 02. Save it locally on laptop or desktop. 03. Move it to Bootstrap node /dcos/dcos_software 04. Make directory /dcos/dcos_software/genconf 05. Create file /dcos/dcos_software/genconf/ config.yaml bootstrap_url: http://<BootStrapNode_IP>:80 cluster_name: 'dcoscluster' exhibitor_storage_backend: static master_discovery: static master_list: - <IP_Address_Master> resolvers: - 10.86.1.1 - 10.86.2.1 - 10.1.1.1 security: disabled oauth_enabled: 'false' use_proxy: 'false' 06. Execute #ifconfig and find out ens32 attached with ip address of the host 07. Create genconf directory on bootstrap node. 08. Create genconf/ip-detect script 09. Create file /dcos/dcos_software/genconf/ip-detect and include following. Make sure ...