A working Kubernetes cluster (v1.10.7 or newer) with Cloud Storage read-write
permissions enabled (https://www.googleapis.com/auth/devstorage.read_write scope)
A working installation of kubectl configured to access the
cluster.
A working installation of gcloud configured to access the Google Cloud
Platform project.
A global static IPv4 address (e.g., static-ip-name).
A DNS A record nexus.example.com pointing to this IPv4 address.
A DNS CNAME record containers.example.com pointing to
nexus.example.com.
The very first thing one must do after deploying Nexus is to log-in into the
Nexus UI with the default credentials (admin:admin123 ) and proceed to change
the password to something more secure. But one must not do it right away!
One will do it after securing Nexus with HTTPS.
The nexus-backup container uses the aforementioned credentials to
access the Nexus API and execute backups. The same credentials are provided
during deployment. Therefore, before deploying Nexus, logging-in and changing
the password as instructed above, one must decide what password will be set
and create a Kubernetes secret containing it, and only then deploy Nexus.
That can be done as follows:
One should allow for 5 to 10 minutes for GCLB to update. Nexus should then
become available over HTTP at http://nexus.example.com.
Securing Nexus with HTTPS
In order to secure Nexus external access one must configure HTTPS access.
The easiest and cheapest way to obtain a trusted TLS certificate is using
Let's Encrypt, and the easiest way to automate the
process of obtaining and renewing certificates from Let's Encrypt is by using
cert-manager:
The easiest way is to install cert-manager via Helm, but a static manifest is available as well.
One can follow these installation instructions.
As soon as it starts, cert-manager will start monitoring Ingress resources and
requesting certificates from Let's Encrypt.
After installation, one will need to setup an issuer and actually request a certificate for Nexus:
Attention: One must edit the certificate.yaml file according to one's setup before running the following commands.
NOTE: Let's Encrypt must be able to reach port 80 on domains for which
certificates are requested, hence the annotation kubernetes.io/ingress.allow-http
in nexus-ingress.yaml must be set to "true".
If everything goes well, after a while one will
be able to access https://nexus.example.com securely and proceed to log-in
into Nexus with the default credentials (admin:admin123), and finally change
the admin password to the secure password one decided above.
Attention: As mentioned in the pre-requisites, the GKE cluster needs read-write
permissions on GCP Cloud Storage in order to upload backups.
The backup procedure uses Google Cloud Storage to save backups. In order to
configure a backup retention policy, one should head over to the backup-lifecycle
directory and install one of the available policies by running:
Attention: As mentioned in the pre-requisites, the GKE cluster needs read-write
permissions on GCP Cloud Storage in order to upload backups.
Backup
Nexus has a built-in
Export databases for backup task which can be used to backup configuration and metadata. However, the generated backup doesn't include blob stores, rendering it semi-useless in a disaster recovery scenario. It is thus of the utmost importance to backup blob stores separately and at roughly the same time this task runs in order to achieve consistent backups.
This is the role of the
nexus-backup
container — a container with a script which makes backups of the default
blob store and then uploads them on a Google Cloud Storage bucket alongside the
data generated by Nexus built-in backup task. These backups are triggered by
a Nexus task of type Execute script (to be configured as described in
docs/admin/configuring-nexus.md).
The script reacts to changes made to
/nexus-data/backup/.backup
and initiates the backup process:
Check if Nexus can be reached. Proceed only if Nexus is reached successfully.
Retrieve the current timestamp in order to tag the backup appropriately.
Make sure that the scripts used to start/stop repositories are installed.
Stop all the repositories so that no writes are made during the backup.
Give Nexus a grace period for the configuration and metadata backup task to
complete.
Archive the default blob store using tar,
streaming the resulting
file to the target bucket.
Archive the backup files generated by Nexus using tar, streaming the
resulting file to the target bucket.
Cleanup all leftover *.bak files in /nexus-data/backup.
Start all the repositories so that Nexus can become available again.
It is advisable to configure this backup procedure to run at off-peak hours, as
described in the aforementioned document.
Restore
In a disaster recovery scenario, the latest backup made by the nexus-backup
container should be restored. In order to achieve this, Nexus must be stopped.
The procedure goes as follows:
$ kubectl exec -i -t nexus-0 --container nexus -- sh # Enter the container.
$ mv /etc/service/nexus/ /nexus-service/ # Prevent `runsvdir` from respawning Nexus after the next step is performed.
$ pkill java # Ask for Nexus to terminate gracefully.
Attention: after terminating Nexus, with the default readiness/liveness probe settings in the chart kubernetes will restart the nexus pod before you've been able to untar the back-up files. If you meet that problem (the sh into the nexus container terminates abruptly with exit code 137), you may overcome it by extending the time limits of the readiness/liveness probes: kubectl edit statefulset nexus and change spec.template.spec.containers.livenessProbe.failureThreshold and spec.template.spec.containers.readinessProbe.failureThreshold for the nexus container to a large value, e.g. 100. Once you're done with the restore, don't forget to reset those values to their initial setting!!
At this point, Nexus is stopped but the container is still running, giving one a
chance to perform the restore procedure.
Attention: One must not close this terminal window just yet.
One should now open another terminal window and login into the nexus-backup container in order to use gsutil to retrieve the desired backup.
Retrieve from GCS the blobstore.tar and databases.tar files
from the last known good backup.
Remove everything under/nexus-data/backup/,
/nexus-data/blobs/default/ and /nexus-data/db/
Run tar -xvf blobstore.tar --strip-components 3 -C /nexus-data/blobs/default/.
Run tar -xvf databases.tar --strip-components 2 -C /nexus-data/restore-from-backup/.
At this point the backup is ready to be restored by Nexus and one can leave the nexus-backup container.
One must now go back to the terminal of the nexus container and run the following commands:
This will automatically start Nexus and perform the restore process.
If one watches the nexus container logs as it boots, one should see an indication
that a restore procedure is in place. After a few minutes access should be
restored with the last known good backup in place.
Ecosystem Role
Standard MoltPulse indexed agent.
Embed Badge
Show off your Pulse Score in your GitHub README to build trust and rank higher.