Volume Management

Shipyard creates snapshots for each named volume in your application so you can easily revert its data. This can be used to fix a commit that introduced data regression, run tests on a specific state, demo a specific scenario, etc.

version: '3.8'

services:
  postgres:
    image: postgres:latest
    environment:
      POSTGRES_DB: app
      POSTGRES_USER: obscure-user
      POSTGRES_PASSWORD: obscure-password
    volumes:
      # this example is using named volume
      - postgres-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    restart: always

volumes:
  # this is how you create a named volume
  postgres-data:

You can visualize the status of your current volumes on the Data Dashboard, manually create snapshots, and freely load them across sibling environments.

note

This is a premium feature. Reach out to us at support@shipyard.build if you would like to try this.

tip

If you use Postgres, there's a great project that makes auto-upgrades super easy. Here's a Shipyard writeup on it, or check out the GitHub repo.

Creating a snapshot

There are 4 types of snapshots that Shipyard creates for your environments:

• Initial : Taken automatically after the first build succeeds
• Post-stop: Taken automatically after a build is retired
• Pre-load: Taken automatically when you manually load a different snapshot onto a running build
• Manual: Triggered by a user via the Data Dashboard

tip

Snapshots do not necessarily have a 1:1 correspondence with builds.

For base environments (i.e., the environments to which your PRs are being merged), snapshots are taken following the logic above.

For pull request environments the same logic applies, with the difference being that we load the most recent Initial or Post-stop snapshot from its base environment before finalizing the environment. In other words, this means that by the time we take the Initial snapshot (after the build succeeds), the data from your base environment is already loaded into the PR environment.

Loading a snapshot

Loading a snapshot is only automatic upon the creation of a new PR environment, in which we load the most recent snapshot from its base environment.

Otherwise, loading a snapshot is a manual process. You can select snapshots from any sibling environments via the Load Snapshot button on the Snapshot History table:

Important distinction

Creating a snapshot ≠ loading a snapshot!

Although sometimes loading a snapshot triggers a snapshot creation, these two actions are distinct and will most likely NOT happen together. For example, let’s say a user does the following:

• Loads a snapshot on their base environment
• Accesses the environment to change something in its data
• Creates a PR, spinning up a new environment

Will the changes the user made be reflected in the new PR?

No, because the new PR environment will take the most recent snapshot from its base environment and no snapshot was taken after the changes in data. Unless you have just loaded a snapshot into an environment, there is no guarantee that the state of your volume will match that of the loaded snapshot.

In other words, if there are changes in the base environment that you want to make sure to propagate to child environments, you’ll need to create a manual snapshot and load it into the PR environment.

Understanding the current status of your volumes

Since the current volume state does not necessarily match with the state of the latest loaded snapshot, our Current Data Volumes table displays two fields to help you understand the state of your current volumes: the Based on and Status fields.

"Based on"

The Based on tooltip shows the loaded snapshot as well as the most recent related snapshot:

The “related snapshot” is a snapshot that has the loaded snapshot as its origin. Take this example:

In this table, snapshot 3 is the origin for two trees of snapshots: 3 → 4 → 5 and 3 → 6. Snapshot 6 is the most recent, so this is the most recent related snapshot.

As a consequence, it also is the snapshot that most closely reflects the current status of your volume. However, it is not guaranteed to be exact, so be sure to create a manual snapshot if you want to save any changes!

Status

There are two main status groups:

1) If there is a load snapshot action, we will display its status ( Mounted, Queued, or Loading)

2) If there is NO load snapshot action, that means your volume is based on the state of your first build, so you’ll see the status Initial.

FAQ

I need to revert the db to an earlier state because I pushed a commit that created a bad migration and now my environment won’t start because of it. Help.

Since snapshots are taken automatically once we retire a build, there should be a snapshot that was taken from a build that was still in a good state. You can find it then load it on the Snapshot History table in the application’s Data Dashboard.

We’ve identified a regression but not until days later. Since other PRs were merged into the base branch, several snapshots were faulty and as a result, new PRs also were loading faulty snapshots.

• What should I do? You should fix the volume state and manually create a snapshot so you can load them to the appropriate environments.
• Can I revert all of them at once? We have a Batch Snapshot Load dashboard that is currently in beta. Please contact support so we can enable it for your organization.
• I created a snapshot with the fixes, is there a way to identify or favorite a snapshot? Although you cannot favorite a snapshot, you can add a note to your snapshot so you can easily identify it.

---

Resources

Volume management with the Shipyard CLI