Payload storage

MicroB2B keeps the content of messages apart from their details. The details live in the database. The content lives in the payload storage: a local directory or an S3 bucket. This page explains how to choose, set up and test it.

Where to set it

You choose the storage under System > Settings. See Settings for the whole page.

The directory is required for local storage. The bucket and region are required for S3.

When a change takes effect

MicroB2B reads the storage settings once, when it starts. A saved change takes effect after a restart.

Until then, the settings page shows the storage in use next to the saved one, with a restart needed badge. The REST API says the same in storage_in_use and restart_needed.

Choosing a backend

Local directory is the default. It needs nothing else. The files are only as safe as the disk:

• back up the directory together with the database (see Backup);
• watch the free disk space.

In the container the working directory is /opt/microb2b, so ./payloads is /opt/microb2b/payloads. See Install.

S3 keeps message content off the server’s disk. Restrict the bucket with a bucket policy. Turn on encryption and versioning as your policy requires.

S3 credentials

MicroB2B never stores S3 credentials as settings, and never shows them. It takes them from the standard AWS sources:

• the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY;
• a shared AWS credentials file;
• a role of the machine or container, where your platform offers one.

Prefer a role over fixed keys where you can.

Test the storage

Test storage on the settings page checks that the storage works. It stores a small test file with the saved settings, reads it back, compares it and deletes it. It reports which step failed and why. The whole test takes at most 10 seconds.

The test uses the saved settings, not what you typed in the form. Save first, then test. It runs with the AWS credentials of the MicroB2B process.

S3-compatible services

MinIO and LocalStack work through the S3 Endpoint setting. With an endpoint set, MicroB2B uses path-style addresses.

For MinIO:

1. Create the bucket. MicroB2B does not create it. For example: mc mb local/microb2b-payloads.

2. Set the credentials in MicroB2B’s environment:

   AWS_ACCESS_KEY_ID=minioadmin
   AWS_SECRET_ACCESS_KEY=minioadmin

3. On System > Settings, set Payload storage to Amazon S3 or S3-compatible, S3 Bucket to microb2b-payloads, S3 Region to us-east-1 and S3 Endpoint to http://minio:9000.

4. Click Update, then Test storage.

5. Restart MicroB2B.

The same with the REST API:

curl -X PUT http://localhost:8080/api/v1/settings \
  -H 'Authorization: Bearer <token>' -H 'Content-Type: application/json' \
  -d '{"storage_backend": "s3", "storage_s3_bucket": "microb2b-payloads",
       "storage_s3_region": "us-east-1", "storage_s3_endpoint": "http://minio:9000"}'

For LocalStack, use the endpoint http://localstack:4566 and any credentials.

How files are laid out

<stage> is unpacked, received, headers or receipt. See Payload stages. The whole message as received is made from headers and received, so it is not stored separately.

MicroB2B creates directories when it needs them.

Changing backends

There is no migration tool. To move from one backend to the other:

1. Save the new settings.

2. Stop MicroB2B.

3. Copy the files so they match the new layout. Add or remove the messages/ prefix. For example, from local to S3:

   aws s3 sync ./payloads/ s3://microb2b-payloads/messages/

4. Start MicroB2B.

The database needs no change.

Errors

See Troubleshooting startup for more.