Gitlab

Introduction

Gitlab is a Dev(Sec)Ops platform to handle all the elements in the software development life cycle.

Main features:

Git repository management
Software development planning
Continuous Integration and Continuous Deployment (CI/CD)

The service can be accessed at [1] with your PIC account.

The groups inside gitlab are not synced with the LDAP groups, if you want to have access to some specific group or project in gitlab you have to request it through the application.

Official Gitlab documentation

The official Gitlab documentation can be found here: https://docs.gitlab.com/

Make sure that the documentation you are viewing applies to the Free (Community Edition) distribution and the version deployed at gitlab.pic.es. You can easily find the version number in the Help menu.

The official documentation can be overwhelming, here are some useful links:

Best practices / recommendations

General rules

Make small and atomic changes Commit / push often.
Use branches for dedicated/long developments
Keep the main branch stable the tests should always pass (yes you should have tests!!)
Write descriptive commit messages Avoid messages like “changes”, “test” or “.”
Adopt a branching strategy gitflow, trunk-based,...
Do code reviews if possible Specially if working in a team

Our contributions

Use git!! If you work alone or in a team, use it!
Do not upload big binary files to a git repository Git is not for data, it is for code. If you still want to do it you can use git LFS (see above).
Use .gitignore to track only relevant files official docs and some examples
Do not track jupyter notebooks (.ipynb) pair them with a script using jupytext
Do not upload confidential data passwords, ssh keys, etc will be there forever

Troubleshooting

Errors in CI/CD jobs

pods XXXX is forbidden: exceeded quota: There's a limit on the amount of resources that can be used simultaneously for CI/CD jobs. This takes into account your own jobs and jobs from other projects. You should be able to retry the pipeline after some period to wait for resources to be freed. If it doesn't work contact the administrator.
ERROR: Preparation failed: couldn't prepare overwrites: invalid build requests specified: the resource "XXXXXX" requested "X" is higher than limit allowed "X": Your job requested more than the allowed resources. Lower down the amount of resources requested.
ERROR: Job failed (system failure): Error in container build: exit code: 137, reason: 'OOMKilled': The job exceeded the memory limit and it was killed. Try increasing the memory requested for you CI/CD job. If this is not enough contact the servicea administrator.

Configure resource limits in CI/CD jobs

Gitlab CI/CD jobs run as pods on one of PIC's kubernetes clusters. Pods can have resource requests and limits. Resource requests are effectively allocated for the pod, which will be killed if it exceeds the resource limits.

Default resources are:

1GiB memory and 0.5 cpu requested
2GiB memory and 1 cpu limit

Default resources can be overwritten by setting environment variables in the job configuration in .gitlab-ci.yml like this:

cicd_job:

 image: python:3.11-slim
 variables:
   KUBERNETES_CPU_REQUEST: "1"
   KUBERNETES_CPU_LIMIT: "2"
   KUBERNETES_MEMORY_REQUEST: "4Gi"
   KUBERNETES_MEMORY_LIMIT: "6Gi"

Maximum resources that can be overwritten are:

6 GiB memory and 2 cpu requested
8 GiB memory and 4 cpu limit