Data persistence with Docker and tensorflow.

https://docs.docker.com/storage/

https://docs.docker.com/storage/volumes/

https://stackoverflow.com/questions/57341455/how-to-persist-my-notebooks-and-data-in-my-docker-image-container

docker run -it -p 8888:8888 -v /home/personal/Documents/tensorflowData:/tf/notebooks tensorflow/tensorflow:latest-gpu-jupyter

Change SOURCE_FOLDER and DEST_FOLDER accordingly (use absolute paths!).

Now if you navigate to localhost:8888 and create a notebook on DEST_FOLDER, it also should be available on SOURCE_FOLDER.

For tensorflow 2, the destination folder is /tf. You can create a folder of your choice, say: notebook, where you will save your file in your jupyter notebook. Those files will be available in the SOURCE_FOLDER you have set.

The changes in tensorflow-tutorials won’t be kept even if you choose it as your destination folder. You have to use another folder as detailed above.

Uninstall docker from Ubuntu

If you installed docker with snap:

simple uninstall:

sudo snap remove docker

and to disable automatic snapshot:

sudo snap remove --purge docker

If after reinstalling docker you have the following error:

bash: /snap/bin/docker: No such file or directory

you need to run

hash docker

and

hash docker-compose

Installation instruction:

https://docs.docker.com/engine/install/ubuntu/

Note that if you installed an old version with snap, you should follow the instruction provided in this article.

https://docs.docker.com/compose/install/

Docker cheatsheet

Create a DockerFile

At the root of your project, create a file name DockerFile, without any file extension. 

In workdir you put the absolute path to your app.


FROM node:12-alpine
WORKDIR /app
COPY . .
RUN yarn install --production
CMD ["node", "/app/src/index.js"]

Build an image

Cd in the directory where your DockerFile is and run:

$sudo docker build -t getting-started .

Don’t forget the “.”  at the end of the command!

getting-started is the name of the app you are creating in docker. It will be the name of the image.

Run the image in a container

$sudo docker run -dp 3000:3000 getting-started

-p 3000:3000 is mapping the internal port 3000 (the 2nd one) to the external port 3000(the first one). So you will be able to access your app on localhost:3000.

Update an image

First you need to remove the container. For that you will need the container id:

you can get it in two different ways:

$sudo docker ps

or

$sudo docker container ls

that gives you the container process id. You can then stop the container:

$sudo docker stop 123456789

and then remove the container:

$sudo docker rm 123456789

You can also do that in one line by using the force option to close and remove the container in one go:

$sudo docker rm -f 123456789 

Sharing docker image:

You’ll need a docker account in hub.docker.com. 

Create a repository.

Once the repository is created, you’ll see a Docker command example to push your project in the repo:

$docker push username/getting-started:tagname

https://docs.docker.com/get-started/04_sharing_app/

Persistency between restarts

To have data persitency between restart, you will need to mount a volume in the container. That volume will be available from the host but also from container that has mounted the same volume.

Create a volume

$sudo docker volume create todo-db

Mount the volume to the container

$sudo docker run -dp 3000:3000 -v todo-db:/etc/todos getting-started

Now your data will persist between session.

Where the data are actually stored on the disk?

Run:

$sudo docker volume inspect todo-db

It will return:

[

    {

        “CreatedAt”: “2019-09-26T02:18:36Z”,

        “Driver”: “local”,

        “Labels”: {},

        “Mountpoint”: “/var/lib/docker/volumes/todo-db/_data”,

        “Name”: “todo-db”,

        “Options”: {},

        “Scope”: “local”

    }

]

The mountpoint is the actual location of the data on the host.

Watch changes on a simple refresh rather than rebuilding the app:

use bind mount. 

Example command:

$docker run -dp 3000:3000 \
    -w /app -v ${PWD}:/app \
    node:12-alpine \
    sh -c "yarn install && yarn run dev"

You can check the logs: 

$docker logs -f <container-id>

Run MySql on a separate container

Why not put it in the same container as your app? It’s because you  might not want to ship your development db with your app. You might also want to scale the front-end and the back-end up differently.

1- create a network:

$sudo docker network create todo-app
$docker run -d \
   --network todo-app --network-alias mysql \
    -v todo-mysql-data:/var/lib/mysql \
    -e MYSQL_ROOT_PASSWORD=secret \
    -e MYSQL_DATABASE=todos \
    mysql:5.7

Pro-tip

You’ll notice we’re using a volume named todo-mysql-data here and mounting it at /var/lib/mysql, which is where MySQL stores its data. However, we never ran a docker volume create command. Docker recognizes we want to use a named volume and creates one automatically for us.

3- confirm the database is up and running:

docker exec -it <mysql-container-id> mysql -p

Sometimes it’s useful to start with a clean slate and remove all Docker containers and even images. Here are some handy shortcuts.

List all containers (only IDs)

$docker ps -aq

Stop all running containers

$docker stop $(docker ps -aq)

Remove all containers

$docker rm $(docker ps -aq)

Remove all images

$docker rmi $(docker images -q)

Dockerize a node app

https://nodejs.org/en/docs/guides/nodejs-docker-webapp/

Run command in the container

Use the command 

docker exec -it <container name> /bin/bash 

to get a bash shell in the container.

Common issues:

Error: “ENOSPC: no space left on device, open ‘/app/yarn-error.log'”

Solution: https://docs.docker.com/engine/reference/commandline/system_prune/

docker system prune

will remove networks and containers which are not in use.

docker system prune --all

will remove all network, containers and images.

You can also run the following with the docker you need running:

docker system prune -a --volumes