Skip to main content

Upgrading to Tator 1.0.0

· 4 min read
Jonathan Takahashi

This blog post will go over the procedure for upgrading a Tator deployment to the upcoming major release, 1.0.0, with most features recently merged with the main branch. This release requires an upgrade to Postgresql 14 and drops the Elasticsearch dependency for standard queries (although it is still used for storing logs). Deprecated bucket definition formats are also removed in this release. This post assumes that the tator deployment is currently running version 0.2.23, was installed with the install script, and is running on microk8s.

caution

This post only applies to migrations from 0.2.23. A fresh install of Tator 1.0.0 does not require this procedure.

Ensure bucket definitions are updated​

Before proceeding, make sure you have updated bucket definitions in values.yaml and updated project specific bucket definitions in the database. These new definitions are supported from version 0.2.22 on, so follow these procedures before continuing with an upgrade to 1.0.0.

Uninstall Tator​

First, uninstall tator by running make clean from the tator directory:

tator$ make clean
kubectl delete apiservice v1beta1.metrics.k8s.io
kubectl delete all --namespace kubernetes-dashboard --all
helm uninstall tator
pod "kubernetes-dashboard-5b5bd8dd4c-cwbqb" deleted
pod "dashboard-metrics-scraper-665b55f67d-wct8m" deleted
service "kubernetes-dashboard" deleted
service "dashboard-metrics-scraper" deleted
deployment.apps "kubernetes-dashboard" deleted
deployment.apps "dashboard-metrics-scraper" deleted
replicaset.apps "kubernetes-dashboard-5b5bd8dd4c" deleted
release "tator" uninstalled
tator$
note

This will not result in any data loss.

Create a bash shell in a tator_postgis container with the Tator data directory mounted:​

docker run -it --rm -v /media/kubernetes_share:/data cvisionai/tator_postgis:$(git rev-parse HEAD) /bin/bash

Install the Postgresql 11 apt package:​

apt update
apt install postgresql-11

Set up the new database directory​

mkdir /data/postgis/db-files-14
chmod 700 /data/postgis/db-files-14
chown postgres /data/postgis/db-files-14

Start the existing database and drop the postgis extension​

su postgres
cd ~
/usr/lib/postgresql/11/bin/pg_ctl -D /data/postgis/db-files -l logfile start
for DB in $(psql -d tator_online -U django -t -c "SELECT datname from pg_database WHERE datname NOT IN ('postgres', 'template0', 'template1')"); do psql -d $DB -c "DROP EXTENSION IF EXISTS postgis CASCADE" -U django; done
/usr/lib/postgresql/11/bin/pg_ctl -D /data/postgis/db-files -l logfile stop

Run the migration:​

initdb -D /data/postgis/db-files-14 -U django
/usr/lib/postgresql/14/bin/pg_ctl -D /data/postgis/db-files-14 -l logfile start
psql -c "ALTER ROLE django WITH LOGIN PASSWORD 'django123'" -U django -d postgres
/usr/lib/postgresql/14/bin/pg_ctl -D /data/postgis/db-files-14 -l logfile stop
pg_upgrade -d /data/postgis/db-files -D /data/postgis/db-files-14 -b /usr/lib/postgresql/11/bin -B /usr/lib/postgresql/14/bin -U django/usr/lib/postgresql/14/bin/pg_ctl -D /data/postgis/db-files-14 -l logfile start
/usr/lib/postgresql/14/bin/vacuumdb -U django --all --analyze-in-stages
/usr/lib/postgresql/14/bin/pg_ctl -D /data/postgis/db-files-14 -l logfile stop

Modify the pg_hba.conf in the new database directory​

vi /media/kubernetes_share/postgis/db-files-14/pg_hba.conf

Then add the following line: host all all all scram-sha-256

If the migration was successful, rename the directories so 14 is the main one​

exit
mv /data/postgis/db-files /data/postgis/db-files-11
mv /data/postgis/db-files-14 /data/postgis/db-files

Reinstall Tator​

First disable migrations in values.yaml:

migrations:
  enable: false

Then from the tator root directory run:

make cluster

After install completes, run migrations manually​

make gunicorn-bash
python3 manage.py makemigrations

Then answer the following questions:

# Did you rename favorite.entityTypeName to favorite.entity_type_name (a CharField)? [y/N] y
# Did you rename favorite.localization_meta to favorite.localization_type (a ForeignKey)? [y/N] y
# Did you rename favorite.state_meta to favorite.state_type (a ForeignKey)? [y/N] y
# Did you rename favorite.meta to favorite.type (a PositiveIntegerField)? [y/N] y
# Did you rename file.meta to file.type (a ForeignKey)? [y/N] y
# Did you rename leaf.meta to leaf.type (a ForeignKey)? [y/N] y
# Did you rename localization.meta to localization.type (a ForeignKey)? [y/N] y
# Did you rename localizationtype.colorMap to localizationtype.color_map (a JSONField)? [y/N] y
# Did you rename media.summaryLevel to media.summary_level (a IntegerField)? [y/N] y
# Did you rename media.meta to media.type (a ForeignKey)? [y/N] y
# Did you rename mediatype.editTriggers to mediatype.edit_triggers (a JSONField)? [y/N] y
# Did you rename section.annotation_bools to section.object_search (a JSONField)? [y/N] N
# Did you rename section.media_bools to section.object_search (a JSONField)? [y/N] N
# Did you rename section.annotation_bools to section.related_object_search (a JSONField)? [y/N] N
# Did you rename section.media_bools to section.related_object_search (a JSONField)? [y/N] N
# Did you rename state.meta to state.type (a ForeignKey)? [y/N] y

Now do the actual migration:

python3 manage.py migrate

Now reenable migrations and update​

migrations:
  enabled: true
make cluster-update