How to create a Devcontainer for your Python project
•
0 likes•108 views
Prevent mis-aligned environments between developers, onboard new-joiners faster, and reduce the time it takes to take your project to production. Sounds interesting? Devcontainers can help you with this. Devcontainers allow you to connect your IDE to a running Docker container and develop inside it. This gives you all the benefits of reproducibility that Docker is known for. In this talk, I will walk you through what Devcontainers are, why they might be useful for you, and how to create one for your Python project using VSCode.
Report
Share
How to create a Devcontainer for your Python project
- 1. How to create a Devcontainer for your Python project 🐳
- 4. The use case Youareassignedtosetupanewrepoforateam.Therequirementsareasfollows: Soweneedtoalignon: 📌 AspecificJavaversion 📌 AspecificPythonversion 📌 Aspecific pyspark version →otherwisewedonotenjoytheguaranteeswewantinproductioncode
- 6. Devcontainers to the rescue ⛑! Dockerhelpsuscreateaformaldefinitionofourenvironment.Devcontainersallowyouto connectyoureditor(IDE)tothatcontainer. 📝 NotethatthisdoesmeanrunningDockerimagesonyourlaptop(performance requirement). Devcontainerscanhelpus: 🔄 Reproducibledevelopmentenvironment ⚡️ Fasterprojectsetup→fasteronboarding 👨👩👧👦 Betteralignmentbetweenteammembers ⏱ Forcedtokeepyourdevenvironmentup-to-date&reproducible →savesyourteamtimegoingintoproductionlater
- 7. 👷🏻♂️Let's build a Devcontainer!
- 8. Let’ssaywehaveareallysimpleprojectthatlookslikethis: $ tree . . ├── README.md ├── requirements.txt ├── requirements-dev.txt ├── sales_analysis.py └── test_sales_analysis.py
- 9. The .devcontainer folder YourDevcontainerspecwillliveinsidethe .devcontainer folder. Therewillbetwomainfiles: devcontainer.json Dockerfile
- 10. Createanewfilecalled devcontainer.json : { "build": { "dockerfile": "Dockerfile", "context": ".." } }
- 11. Sohowdoesthis Dockerfile looklike? FROM python:3.10 # Install Java RUN apt update && apt install -y sudo && sudo apt install default-jdk -y ## Pip dependencies # Upgrade pip RUN pip install --upgrade pip # Install production dependencies COPY requirements.txt /tmp/requirements.txt RUN pip install -r /tmp/requirements.txt && rm /tmp/requirements.txt # Install development dependencies COPY requirements-dev.txt /tmp/requirements-dev.txt RUN pip install -r /tmp/requirements-dev.txt && rm /tmp/requirements-dev.txt
- 12. Opening the Devcontainer The .devcontainer folderinplace,nowit’stimetoopenourDevcontainer.
- 13. Openupthecommandpallete( CMD + Shift + P )andselect“DevContainers:Reopen inContainer”:
- 16. What is happening under the hood 🚗 BesidesstartingtheDockerimageandattachingtheterminaltoit,VSCodeisdoinga couplemorethings: 1. isbeinginstalledonyourDevcontainer. VSCodeServerisinstalledasaserviceinthecontaineritselfsoyourVSCode installationcancommunicatewiththecontainer. Forexample,installandrunextensions. VSCodeServer 2.Configiscopiedover. Configlike ~/.gitconfig and ~/.ssh/known_hosts arecopiedovertotheir respectivelocationsinthecontainer. 3.Filesystemmounts. VSCodeautomaticallytakescareofmounting: ThefolderyouarerunningtheDevcontainerfrom. YourVSCodeworkspacefolder.
- 17. Opening your Devcontainer with the click of a button YourentireprojectsetupisnowencapsulatedintheDevcontainer.Soactuallywecanadd aMarkdownbuttontoopenuptheDevcontainer: JustmodifytheGitHubURLafter url= ✓. [  ]( https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume? url=https://github.com/godatadriven/python-devcontainer-template )
- 18. Thisrendersthefollowingbutton: Remote - Containers Remote - Containers Open Open
- 22. Installing a non-root user Ifyou pip install anewpackage,youwillseethefollowingmessage:
- 23. Solet'sgoaheadandcreateauserforthisscenario. # Add non-root user ARG USERNAME=nonroot RUN groupadd --gid 1000 $USERNAME && useradd --uid 1000 --gid 1000 -m $USERNAME ## Make sure to reflect new user in PATH ENV PATH="/home/${USERNAME}/.local/bin:${PATH}" USER $USERNAME Addthefollowingpropertyto devcontainer.json : "remoteUser": "nonroot" That'sgreat!Whenwenowstartthecontainerweshouldconnectastheuser nonroot .
- 24. Passing custom VSCode settings "customizations": { "vscode": { "extensions": [ "ms-python.python" ], "settings": { "python.testing.pytestArgs": [ "." ], "python.testing.unittestEnabled": false, "python.testing.pytestEnabled": true, "python.formatting.provider": "black", "python.linting.mypyEnabled": true, "python.linting.enabled": true } } }
- 25. Accessing Spark UI Sinceweareusingpyspark,itwouldbenicetobeabletoaccessSparkUI. "portsAttributes": { "4040": { "label": "SparkUI", "onAutoForward": "notify" } }, "forwardPorts": [ 4040 ]
- 28. ✨
- 29. Running our CI in the Devcontainer Therearetwobasicoptions: 1.BuildtheDockerimagewithintheCI/CDpipeline 2.Prebuildingtheimage Let'sseeaboutoptionnumber(1).
- 30. 1. Build the Docker image within the CI/CD pipeline Luckily,aGitHubActionwasalreadysetupforustodoexactlythis: devcontainers/ci
- 31. Tonowbuild,pushandrunacommandintheDevcontainerisaseasyas: name: Python app on: ... jobs: build: runs-on: ubuntu-latest steps: - name: Checkout (GitHub) uses: actions/checkout@v3 - name: Login to GitHub Container Registry uses: docker/login-action@v2 with: registry: ghcr.io username: ${{ github.repository_owner }} password: ${{ secrets.GITHUB_TOKEN }} - name: Build and run dev container task uses: devcontainers/ci@v0.2 with: imageName: ghcr.io/${{ github.repository }}/devcontainer runCmd: pytest .
- 33. Awesome!
- 34. The final Devcontainer definition WebuiltthefollowingDevcontainerdefinitions.
- 35. First, devcontainer.json : { "build": { "dockerfile": "Dockerfile", "context": ".." }, "remoteUser": "nonroot", "customizations": { "vscode": { "extensions": [ "ms-python.python" ], "settings": { "python.testing.pytestArgs": [ "." ], "python.testing.unittestEnabled": false, "python.testing.pytestEnabled": true, "python.formatting.provider": "black", "python.linting.mypyEnabled": true, "python.linting.enabled": true } } }, ... }
- 36. "portsAttributes": { "4040": { "label": "SparkUI", "onAutoForward": "notify" } }, "forwardPorts": [ 4040 ]
- 37. Andour Dockerfile : FROM python:3.10 # Install Java RUN apt update && apt install -y sudo && sudo apt install default-jdk -y # Add non-root user ARG USERNAME=nonroot RUN groupadd --gid 1000 $USERNAME && useradd --uid 1000 --gid 1000 -m $USERNAME ## Make sure to reflect new user in PATH ENV PATH="/home/${USERNAME}/.local/bin:${PATH}" USER $USERNAME ## Pip dependencies # Upgrade pip RUN pip install --upgrade pip # Install production dependencies COPY --chown=nonroot:1000 requirements.txt /tmp/requirements.txt RUN pip install -r /tmp/requirements.txt && rm /tmp/requirements.txt # Install development dependencies COPY --chown=nonroot:1000 requirements-dev.txt /tmp/requirements- dev.txt RUN pip install -r /tmp/requirements-dev.txt && rm /tmp/requirements-dev.txt
- 39. Going further 🔮 Devcontainerfeatures Devcontainertemplates 💡 Protip:mountyourAWS/GCP/Azurecredentials ...andmuchmore: Mountingdirectories
- 40. Awesome resources .RunyourCIinyourDevcontainers.Builtonthe . .TheofficialDevcontainerspecification. .Acollectionofready-to-useDevcontainerimages. .Moreexplanations&instructionsforaddinga non-rootusertoyour Dockerfile and devcontainer.json . .Arepopointingtoyetevenmoreawesomeresources. devcontainers/ci DevcontainerCLI https://containers.dev/ devcontainers/images Addanon-rootusertoacontainer Pre-buildingdevcontainerimages awesome-devcontainers
- 41. Concluding Devcontainershaveprovenusefulto: 🔄 Reproducibledevelopmentenvironment ⚡️ Fasterprojectsetup→fasteronboarding 👨👩👧👦 Betteralignmentbetweenteammembers ⏱ Forcedtokeepyourdevenvironmentup-to-date&reproducible →savesyourteamtimegoingintoproductionlater NowonlyVSCode,but takingshape. openspecification