chore(deps): upgrade to Rails 7.2.3.2, Ruby 3.4.10 and latest gems

Upgrades Rails from 7.1.6 to 7.2.3.2, Ruby from 3.4.6 to 3.4.10, and
brings the rest of the bundle up to date.

Two things were broken by Rails 7.2 and are fixed here:

* MigrationWaiter called `ActiveRecord::Base.connection.migration_context`,
  which moved to the connection pool in 7.2. This raised NoMethodError on
  boot for any deployment with `migration_waiter.enabled` set, and had no
  test covering it.
* `config.action_dispatch.show_exceptions = false` was deprecated in 7.1
  and removed in 7.2. Now `:none`.

`json` is pinned to < 3. Active Support 7.2 passes `quirks_mode:` to both
`JSON.generate` and `JSON.parse`, and json 3.0 removed that option, so an
unpinned json resolves to 3.x and makes every `to_json` raise - webhook
payloads included. Nothing else in the bundle constrains it. The pin can go
when we reach Rails 8.1, which no longer passes the option.

`dynamic_form` is dropped. It is unmaintained and from 1.3.1 requires Ruby
< 3.4, so it could never be updated again. Ten views used it, so
`error_messages_for` now lives in ErrorMessagesHelper and `f.error_messages`
in a form builder extension. The rendered markup is unchanged apart from a
heading and description the stylesheet had always hidden; the rules that
hid them are removed too.

`execjs` is unpinned. The `< 2.8` constraint dated from the Rails 6.0
upgrade and asset output is byte-identical without it.

Notable gem majors: puma 8, haml 7, secure_headers 7, sentry 7, jwt 3,
prometheus-client 5, highline 3, omniauth-rails_csrf_protection 2,
rspec-rails 8, shoulda-matchers 8, rubocop 1.91. secure_headers 7 changes
X-XSS-Protection from "1; mode=block" to "0", which is the correct modern
default and matches Rails' own.

`config.load_defaults` deliberately stays at 7.0. The pending 7.1 and 7.2
options are listed, commented out, in new_framework_defaults_7_1.rb and
_7_2.rb so they can be adopted in stages - several change the signed cookie
and cache formats and need a staged deploy. The vestigial
new_framework_defaults_7_0.rb is removed, as `load_defaults 7.0` already
applies all of it.

Adds tests for the areas this touched, which were mostly uncovered:
MigrationWaiter, the serialized columns that gate a defaults bump, JSON
encoding, HAML template compilation for all 103 templates, page rendering,
form error rendering and security headers. The assets Gemfile group now
includes :test so the view layer can be exercised at all.

.github/ISSUE_TEMPLATE/bug_report.md

@@ -2,7 +2,7 @@
 name: 🐛 Bug report
 about: Create a report to help us improve Postal and fix issues.
 title: ''
-labels: bug
+labels: ''
 assignees: ''
 
 ---

.github/workflows/ci.yml

@@ -3,63 +3,134 @@ name: CI
 on: [push]
 
 jobs:
-  # lint:
-  #   name: Lint
-  #   runs-on: self-hosted
-  #   steps:
-  #     - uses: actions/checkout@v2
-  #     - run: make ci-lint
+  release-please:
+    runs-on: ubuntu-latest
+    if: github.ref == 'refs/heads/main'
+    outputs:
+      release_created: ${{ steps.release-please.outputs.release_created }}
+      tag_name: ${{ steps.release-please.outputs.tag_name }} # e.g. v1.0.0
+      version: ${{ steps.release-please.outputs.version }} # e.g. 1.0.0
+      all: ${{ toJSON(steps.release-please.outputs) }}
+    steps:
+      - uses: google-github-actions/release-please-action@v3
+        id: release-please
+        with:
+          command: manifest
+
+  build:
+    name: CI Image Build
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: docker/setup-buildx-action@v2
+      - uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - uses: docker/build-push-action@v4
+        with:
+          push: true
+          tags: ghcr.io/postalserver/postal:ci-${{ github.sha }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+          target: ci
+          platforms: linux/amd64
 
   test:
-    name: Test
-    runs-on: ubuntu-20.04
+    name: Test Suite
+    runs-on: ubuntu-latest
+    needs: build
     steps:
-      - uses: actions/checkout@v2
-      - name: Run tests
-        run: make ci-test
-
-  build-latest:
-    runs-on: ubuntu-20.04
-    needs: [test]
-    if: github.ref == 'refs/heads/main'
-    steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0
-      - name: Parse git ref
-        run: |
-          echo "DOCKER_TAG_NAME=$(basename $GITHUB_REF)" >> $GITHUB_ENV
-
-      - name: Log in to the Container registry
-        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
+      - uses: docker/login-action@v2
         with:
           registry: ghcr.io
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Build and push Docker image
-        run: make docker-release
-
-  build-release:
-    runs-on: ubuntu-20.04
-    needs: [test]
-    if: contains(github.ref, 'refs/tags/')
-    steps:
-      - uses: actions/checkout@v2
-        with:
-          fetch-depth: 0
-      - name: Parse git ref
-        run: |
-          echo "DOCKER_TAG_NAME=$(basename $GITHUB_REF)" >> $GITHUB_ENV
-
-      - name: Log in to the Container registry
-        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
-        with:
-          registry: ghcr.io
-          username: ${{ github.actor }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Build and push Docker image
-        run: make docker-release
+      - run: docker compose pull
         env:
-          DOCKER_TAG_NAME: "latest"
+          POSTAL_IMAGE: ghcr.io/postalserver/postal:ci-${{ github.sha }}
+      - run: docker compose run postal sh -c 'bundle exec rspec'
+        env:
+          POSTAL_IMAGE: ghcr.io/postalserver/postal:ci-${{ github.sha }}
+
+  release-branch:
+    name: Release (branch)
+    runs-on: ubuntu-latest
+    needs: [build]
+    if: >-
+      startsWith(github.ref, 'refs/heads/') &&
+      startsWith(github.ref, 'refs/heads/release-please-') != true &&
+      startsWith(github.ref, 'refs/heads/dependabot/') != true
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: docker/setup-qemu-action@v3
+      - uses: docker/setup-buildx-action@v2
+      - uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - id: info
+        run: |
+          IMAGE=ghcr.io/postalserver/postal
+
+          REF="${GITHUB_REF#refs/heads/}"
+          if [ -z "$REF" ]; then exit 1; fi
+
+          VER="$(git describe --tags 2>/dev/null)"
+          echo "version=${VER}" >> "$GITHUB_OUTPUT"
+          echo "branch=${REF}" >> "$GITHUB_OUTPUT"
+
+          echo 'tags<<EOF' >> "$GITHUB_OUTPUT"
+          if [[ "$REF" == "main" ]]; then
+            echo "${IMAGE}:latest" >> "$GITHUB_OUTPUT"
+          else
+            echo "${IMAGE}:branch-${REF}" >> "$GITHUB_OUTPUT"
+          fi
+          echo 'EOF' >> "$GITHUB_OUTPUT"
+      - uses: docker/build-push-action@v4
+        with:
+          push: true
+          tags: ${{ steps.info.outputs.tags }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+          target: full
+          platforms: linux/amd64
+          build-args: |
+            VERSION=${{ steps.info.outputs.version }}
+            BRANCH=${{ steps.info.outputs.branch }}
+
+  publish-image:
+    name: Publish Image
+    runs-on: ubuntu-latest
+    needs: [build, test, release-please]
+    if: ${{ needs.release-please.outputs.release_created }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: docker/setup-qemu-action@v3
+      - uses: docker/setup-buildx-action@v2
+      - uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - uses: docker/build-push-action@v4
+        with:
+          push: true
+          tags: |
+            ghcr.io/postalserver/postal:stable
+            ghcr.io/postalserver/postal:${{ needs.release-please.outputs.version }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+          target: full
+          build-args: |
+            VERSION=${{ needs.release-please.outputs.version }}
+          platforms: linux/amd64

.github/workflows/close.yml

@@ -0,0 +1,30 @@
+name: 'Close stale issues and PRs'
+on:
+  schedule:
+    - cron: '30 1 * * *'
+  workflow_dispatch: 
+permissions:
+  issues: write
+  pull-requests: write
+jobs:
+  stale:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/stale@v9
+        with:
+          stale-issue-message: 'This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.'
+          stale-pr-message: 'This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.'
+          close-issue-message: 'This issue was closed because it has been stalled for 5 days with no activity.'
+          close-pr-message: 'This PR was closed because it has been stalled for 10 days with no activity.'
+
+          stale-issue-label: stale
+          stale-pr-label: stale
+          
+          days-before-issue-stale: 30
+          days-before-pr-stale: 45
+          days-before-issue-close: 5
+          days-before-pr-close: 10
+
+          exempt-all-assignees: true
+          exempt-all-milestones: true
+          exempt-issue-labels: bug,enhancement,docs,install,feature

.gitignore

@@ -29,4 +29,11 @@ vendor/bundle
 
 Procfile.local
 VERSION
+BRANCH
+
+.rubocop-https*
+.env*
+
+node_modules
+yarn.lock
 

.release-please-manifest.json

@@ -0,0 +1,3 @@
+{
+  ".": "3.3.7"
+}

.rubocop.yml

@@ -0,0 +1,179 @@
+AllCops:
+  TargetRubyVersion: 3.4
+  NewCops: enable
+  Exclude:
+    - "bin/*"
+    - "db/schema.rb"
+    # Fixes missing gem exception when running Rubocop on GitHub Actions.
+    - "vendor/bundle/**/*"
+    - lib/tasks/auto_annotate_models.rake
+
+# Always use double quotes
+Style/StringLiterals:
+  EnforcedStyle: double_quotes
+  AutoCorrect: true
+
+# We prefer arrays of symbols to look like an array of symbols.
+# For example: [:one, :two, :three] as opposed to %i[one two three]
+Style/SymbolArray:
+  EnforcedStyle: brackets
+
+# There should always be empty lines inside a class. For example
+#
+#    class MyExample
+#
+#      def some_method
+#      end
+#
+#    end
+Layout/EmptyLinesAroundClassBody:
+  EnforcedStyle: empty_lines
+
+# We want to keep attr_* definitions separated on their own lines, rather than
+# all of them collapsed into a single attr_* call. The collapsed/grouped variant
+# is harder to read, and harder to see what's been changed in diffs.
+Style/AccessorGrouping:
+  Enabled: false
+
+# Blocks are slightly different to classes, in these cases there should
+# not be new lines around the contents of the block.
+#
+#    proc do
+#      # Do something
+#    end
+Layout/EmptyLinesAroundBlockBody:
+  EnforcedStyle: no_empty_lines
+
+# Modules are the same as classes unless they're being used for namespacing
+# purposes in which case there should not be new lines.
+Layout/EmptyLinesAroundModuleBody:
+  EnforcedStyle: empty_lines_except_namespace
+
+# Space is required following -> when writing a lambda:
+#
+#    somethign = -> (var) { block }
+Layout/SpaceInLambdaLiteral:
+  EnforcedStyle: require_space
+
+Layout/FirstHashElementIndentation:
+  Enabled: false
+
+# We don't mind setting assignments in conditions so this has been disabled to
+# allow `if something = something_else` without worrying about brackets.
+Lint/AssignmentInCondition:
+  Enabled: false
+
+# Top level documentation is quite rare...
+Style/Documentation:
+  Enabled: false
+
+# We want to allow inner slashes in a regexp to be used when using /xxx/ form.
+Style/RegexpLiteral:
+  AllowInnerSlashes: true
+
+# Blocks of if statements are perfectly fine and usually more readable than
+# putting everything onto a single line just because we can.
+Style/IfUnlessModifier:
+  Enabled: false
+
+# We prefer assignments to happen within the condition rather than setting a
+# variable to the result of a condition.
+Style/ConditionalAssignment:
+  EnforcedStyle: assign_inside_condition
+  IncludeTernaryExpressions: false
+
+# Empty methods should not be compacted onto a single line
+Style/EmptyMethod:
+  EnforcedStyle: expanded
+
+# As above, just flag them.
+Lint/UnusedBlockArgument:
+  AutoCorrect: false
+
+# While we don't want to make heavy use of get_ or set_ methods we do often need
+# to use these when we want to refer to actually getting or setting something
+# (usually from another API).
+Naming/AccessorMethodName:
+  Enabled: false
+
+# If we want a boolean called :true, we should be allowed that. These are likely
+# not mistakes.
+Lint/BooleanSymbol:
+  Enabled: false
+
+# Using block.map(&:upcase) is not always the neatest way to show something. For
+# example if you have a block that just calls one thing, you don't want it
+# collapsed.
+#
+#    action do |user|
+#      user.delete
+#    end
+#
+# This should be action(&:delete) because it is not clear what is actually
+# happening without the context of knowing what the inner variable should be
+# called.
+Style/SymbolProc:
+  Enabled: false
+
+# Allow a maxmium of 5 arguments and don't include keyword arguments
+Metrics/ParameterLists:
+  Max: 5
+  CountKeywordArgs: false
+
+# This cop checks for chaining of a block after another block that spans multiple lines.
+Style/MultilineBlockChain:
+  Exclude:
+    - "spec/**/*.rb"
+
+Style/TrailingCommaInArrayLiteral:
+  EnforcedStyleForMultiline: consistent_comma
+
+Metrics/AbcSize:
+  Enabled: false
+
+Style/FrozenStringLiteralComment:
+  Enabled: true
+  SafeAutoCorrect: true
+
+Naming/PredicatePrefix:
+  Enabled: false
+
+Layout/LineLength:
+  # We want to reduce this back down to 120 but there are a fair number of offences
+  # of this which need addressing individually and carefully. 
+  Max: 200
+
+Metrics/PerceivedComplexity:
+  # As above, we want to enable this again in the future, but for now we'll just 
+  # disable it entirely.
+  Enabled: false
+
+Metrics/CyclomaticComplexity:
+  # As above.
+  Enabled: false
+
+Metrics/MethodLength:
+  # As above.
+  Enabled: false
+
+Metrics/BlockNesting:
+  # As above.
+  Enabled: false
+
+Style/StringConcatenation:
+  Enabled: false
+
+Metrics/BlockLength:
+  Enabled: false
+
+Metrics/ClassLength:
+  Enabled: false
+
+Metrics/ModuleLength:
+  Enabled: false
+
+Lint/UnusedMethodArgument:
+  Enabled: false
+
+Style/SpecialGlobalVars:
+  Enabled: false

.ruby-version

@@ -1,2 +1 @@
-2.6.9
-
+3.4.10

.schmersion.yaml

@@ -1,14 +0,0 @@
-exports:
-  - name: CHANGELOG.md
-    formatter: markdown
-    options:
-      title: CHANGELOG
-      description: This file contains all the latest changes and updates to Postal.
-      sections:
-        - title: Features
-          types: [feat]
-        - title: Bug Fixes
-          types: [fix]
-#
-# There are additional options you can define, check out the README for details.
-# https://github.com/krystal/schmersion

CHANGELOG.md

@@ -2,6 +2,493 @@
 
 This file contains all the latest changes and updates to Postal.
 
+## [3.3.7](https://github.com/postalserver/postal/compare/3.3.6...3.3.7) (2026-06-03)
+
+
+### Bug Fixes
+
+* **http:** prevent SSRF in outbound webhook and HTTP endpoint requests ([11c9814](https://github.com/postalserver/postal/commit/11c9814474f956619da35e8385ef7fab9f304de0))
+* **message-db:** prevent SQL injection via condition keys (GHSA-x2hq-rfpg-3xr5) ([4314a6e](https://github.com/postalserver/postal/commit/4314a6ec1e2812daa67dd20effd1db1769c1f8e8))
+* **specs:** stub IPv6 support in address guard IPv6 literal spec ([029bfe0](https://github.com/postalserver/postal/commit/029bfe098d9b8c0b5cafc49eac33e767f5748cd3))
+
+
+### Miscellaneous Chores
+
+* **deps:** upgrade rack & rails ([0445e5c](https://github.com/postalserver/postal/commit/0445e5c509870dfe9c16366c53dee3fc02ad3904))
+* update security vulnerability reporting instructions ([8be1e27](https://github.com/postalserver/postal/commit/8be1e27fec489ab659ef5e909f705932028b1694))
+
+## [3.3.6](https://github.com/postalserver/postal/compare/3.3.5...3.3.6) (2026-04-28)
+
+
+### Bug Fixes
+
+* **messages:** sandbox rendered email HTML as extra XSS defence ([cad2aa6](https://github.com/postalserver/postal/commit/cad2aa6808519a3ff25215f09f4966d9fa3bb372))
+
+
+### Miscellaneous Chores
+
+* ignore node modules and yarn.lock ([b611d57](https://github.com/postalserver/postal/commit/b611d577af79b8e1e75b6d47fa04d1ba03e34eec))
+
+
+### Code Refactoring
+
+* **auth:** tighten return_to validation ([84f4e20](https://github.com/postalserver/postal/commit/84f4e20f05db2d11b0144f95960c956f8221e657))
+* **helpers:** escape interpolated values in select options ([9243524](https://github.com/postalserver/postal/commit/924352403553dcfcc569876ca76c219493fac9d6))
+* **tracking:** remove unused src image proxy ([dca7f90](https://github.com/postalserver/postal/commit/dca7f90b9046247c0d953567be35921167e79d87))
+
+## [3.3.5](https://github.com/postalserver/postal/compare/3.3.4...3.3.5) (2026-02-01)
+
+
+### Bug Fixes
+
+* **deliveries:** escape delivery details to prevent HTML injection ([11419f9](https://github.com/postalserver/postal/commit/11419f99140e13688a9613cab3ee03f8d3cbae45))
+* **health_server:** use rackup handler instead of rack handler ([7c47422](https://github.com/postalserver/postal/commit/7c47422c865e738c4d6af0fed1cca4405288341f))
+* oidc scopes are invalid when concatenated ([#3332](https://github.com/postalserver/postal/issues/3332)) ([9c5f96a](https://github.com/postalserver/postal/commit/9c5f96ae90cf06dcd5db776806865752f667bd95))
+* typo in process logging ([#3212](https://github.com/postalserver/postal/issues/3212)) ([b7e5232](https://github.com/postalserver/postal/commit/b7e5232e077b3c9b7a999dcb6676fba0ec61458e))
+* typo in the credentials page ([fd3c7cc](https://github.com/postalserver/postal/commit/fd3c7ccdf6dc4ee0a76c9523cbd735159e4b8000))
+* update url for v2 config ([#3225](https://github.com/postalserver/postal/issues/3225)) ([e00098b](https://github.com/postalserver/postal/commit/e00098b8003cf37f2708f536871b3ade377aed2d))
+
+
+### Documentation
+
+* **process.rb:** add help about time unit used by metric ([#3339](https://github.com/postalserver/postal/issues/3339)) ([f5325c4](https://github.com/postalserver/postal/commit/f5325c49ff1152ad53eaaec98717ad3412d379ae))
+
+
+### Miscellaneous Chores
+
+* **deps:** upgrade puma, net-imap and other deps ([c03c44b](https://github.com/postalserver/postal/commit/c03c44b442a29aa9881c1e1aae60bead9776a6b6))
+* **dockerfile:** reduce container size ([86de372](https://github.com/postalserver/postal/commit/86de372382bd62bdd5d1372254f8817b0360bd56))
+* remove version from docker-compose.yml ([c78000c](https://github.com/postalserver/postal/commit/c78000ca8f2998aa04648f465060768db6467de6))
+* upgrade resolv to 0.6.2 ([d00d978](https://github.com/postalserver/postal/commit/d00d978872a96369544303d08f6a9d11cdf56b62))
+* upgrade to rails 7.1 and ruby 3.4 ([#3457](https://github.com/postalserver/postal/issues/3457)) ([ab6d443](https://github.com/postalserver/postal/commit/ab6d4430baa33a05f1aa66e776cc2a5bcaa0ede8))
+* upgrade uri gem to 1.0.3 ([f193b8e](https://github.com/postalserver/postal/commit/f193b8e77fc096382ab7aaa6a2c29641b4cb12df))
+
+## [3.3.4](https://github.com/postalserver/postal/compare/3.3.3...3.3.4) (2024-06-20)
+
+
+### Bug Fixes
+
+* fix `postal version` command ([4fa88ac](https://github.com/postalserver/postal/commit/4fa88acea0dececd0eae485506a2ad8268fbea59))
+* fix issue running message pruning task ([3a33e53](https://github.com/postalserver/postal/commit/3a33e53d843584757bb00898746aa059d7616db4))
+* raise NotImplementedError when no call method on a scheduled task ([2b0919c](https://github.com/postalserver/postal/commit/2b0919c1454eabea93db96f50ecbd8e36bb89f1f))
+
+## [3.3.3](https://github.com/postalserver/postal/compare/3.3.2...3.3.3) (2024-04-18)
+
+
+### Bug Fixes
+
+* **legacy-api:** allow _expansions to be provided as true to return all expansions ([39f704c](https://github.com/postalserver/postal/commit/39f704c256fc3e71a1dc009acc77796a1efffead)), closes [#2932](https://github.com/postalserver/postal/issues/2932)
+
+## [3.3.2](https://github.com/postalserver/postal/compare/3.3.1...3.3.2) (2024-03-21)
+
+
+### Code Refactoring
+
+* **versioning:** improve how current version and branch is determined and set ([07c6b31](https://github.com/postalserver/postal/commit/07c6b317f2b9dc04b6a8c88df1e6aa9e54597504))
+
+## [3.3.1](https://github.com/postalserver/postal/compare/3.3.0...3.3.1) (2024-03-21)
+
+
+### Bug Fixes
+
+* **smtp-sender:** ensure relays without a host are excluded ([3a56ec8](https://github.com/postalserver/postal/commit/3a56ec8a74950e0162d98f1af5f58a67a82d6455))
+* **smtp-sender:** fixes `SMTPSender.smtp_relays` ([b3264b9](https://github.com/postalserver/postal/commit/b3264b942776e254d3c351c94c435d172a514e18))
+
+
+### Miscellaneous Chores
+
+* **container:** add the branch name to the container ([bee5098](https://github.com/postalserver/postal/commit/bee509832edc151d97fe5bfc48c4973452873fc8))
+* **github-actions:** don't generate commit- tags ([d65bbe0](https://github.com/postalserver/postal/commit/d65bbe0579037c5df962a18134bc007f5159d7e5))
+* **github-actions:** don't run for dependabot or release-please PRs and fetch whole repo ([adaf2b0](https://github.com/postalserver/postal/commit/adaf2b07502e9ed91290873ad8465051c6fd814f))
+* **github-actions:** include a version string on branch-*/latest images ([64bc7dc](https://github.com/postalserver/postal/commit/64bc7dcf7c0a8e006ab6eb6e8b4a52ad5e7e6528))
+* **ui:** display branch in footer if present ([1823617](https://github.com/postalserver/postal/commit/18236171ebc398c157f2e61b15c7df9f91205284))
+
+
+### Code Refactoring
+
+* remove moonrope but maintain legacy API actions ([#2889](https://github.com/postalserver/postal/issues/2889)) ([4d9654d](https://github.com/postalserver/postal/commit/4d9654dac47d59c760be96388d0421de74d3e6ac))
+
+## [3.3.0](https://github.com/postalserver/postal/compare/3.2.2...3.3.0) (2024-03-18)
+
+
+### Features
+
+* **prometheus:** add `postal_message_queue_latency` metric ([ee8d829](https://github.com/postalserver/postal/commit/ee8d829a854f91e476167869cafe35c2d37bb314))
+* **worker:** allow number of threads to be configured ([7e2accc](https://github.com/postalserver/postal/commit/7e2acccd1ebd80750a3ebdb96cb5c36b5263cc24))
+* **worker:** scale connection pool with worker threads ([ea542a0](https://github.com/postalserver/postal/commit/ea542a0694b3465b04fd3ebc439837df414deb1e))
+
+
+### Bug Fixes
+
+* **message-dequeuer:** ability to disable batching ([4fcb9e9](https://github.com/postalserver/postal/commit/4fcb9e9a2e34be5aa4bdf13f0529f40e564b72b4))
+
+
+### Miscellaneous Chores
+
+* **config-docs:** update docs for latest oidc defaults ([364eba6](https://github.com/postalserver/postal/commit/364eba6c5fce2f08a36489f42856ad5024a2062c))
+* **config-docs:** update proxy protocol to mention v1 ([45dd8aa](https://github.com/postalserver/postal/commit/45dd8aaac56f15481cb7bf9081401cb28dc1e707))
+
+## [3.2.2](https://github.com/postalserver/postal/compare/3.2.1...3.2.2) (2024-03-14)
+
+
+### Bug Fixes
+
+* don't use authentication on org & server deletion ([be45652](https://github.com/postalserver/postal/commit/be456523dd3aacb5c3eb45c9261da97ebffe603c))
+* **smtp-server:** fixes proxy protocol ([9240612](https://github.com/postalserver/postal/commit/92406129cfcf1a06499a6f5aa18c73f1d6195793))
+
+
+### Miscellaneous Chores
+
+* allow config location message to be suppressed ([f760cdb](https://github.com/postalserver/postal/commit/f760cdb5a1d53e9c30ee495d129cbf12603a3cbd))
+* hide further config messages ([1c67f72](https://github.com/postalserver/postal/commit/1c67f72209c93404d7024ce3d15f6f54f2d707c4))
+* suppress config location on default-dkim-record ([aa76aae](https://github.com/postalserver/postal/commit/aa76aae2322af41af1bd60cfe1d69a11ac76324e))
+
+
+### Tests
+
+* add tests for the legacy API ([3d208d6](https://github.com/postalserver/postal/commit/3d208d632f4fc8a4adbfdb2bf4b377271eae6692))
+
+## [3.2.1](https://github.com/postalserver/postal/compare/3.2.0...3.2.1) (2024-03-13)
+
+
+### Bug Fixes
+
+* fixes `postal default-dkim-record` ([58dddeb](https://github.com/postalserver/postal/commit/58dddebeb81dc6fab945d2b10a91588eddc471c2))
+
+## [3.2.0](https://github.com/postalserver/postal/compare/3.1.1...3.2.0) (2024-03-13)
+
+
+### Features
+
+* add sha256 signatures to outgoing http requests ([#2874](https://github.com/postalserver/postal/issues/2874)) ([96d7365](https://github.com/postalserver/postal/commit/96d73653d7cb4dde1fbe74ccb3596147ef8cd9ed))
+* automatically remove queued messages with stale locks ([#2872](https://github.com/postalserver/postal/issues/2872)) ([d84152e](https://github.com/postalserver/postal/commit/d84152eb5df6f963426d6ba8d02d39b3c146c8a5))
+* openid connect support ([#2873](https://github.com/postalserver/postal/issues/2873)) ([5ed94f6](https://github.com/postalserver/postal/commit/5ed94f6f855735aa00544b2574dfb9e65d559a38))
+
+
+### Bug Fixes
+
+* **smtp-server:** add additional information to cram-md5 log entries ([9982bb8](https://github.com/postalserver/postal/commit/9982bb8c31ee4885d188666e2e8afdc218528df7))
+
+
+### Styles
+
+* **rubocop:** Style/TrailingCommaInArrayLiteral ([4e13577](https://github.com/postalserver/postal/commit/4e13577891dc827244abc6bf6d9ab4ee45860556))
+
+
+### Miscellaneous Chores
+
+* regenerate config docs ([5d8213a](https://github.com/postalserver/postal/commit/5d8213a98735f07fdf1700c7d01597654f41dbd0))
+
+## [3.1.1](https://github.com/postalserver/postal/compare/3.1.0...3.1.1) (2024-03-08)
+
+
+### Bug Fixes
+
+* don't override paths in dockerfile ([9399e32](https://github.com/postalserver/postal/commit/9399e3223467cdacd010e70b58ad6093e128213d))
+
+
+### Tests
+
+* **smtp-sender:** add more tests for AUTH LOGIN ([22dcd49](https://github.com/postalserver/postal/commit/22dcd4901f188915cf4b3c758c6f2fc637a4e1e3))
+
+## [3.1.0](https://github.com/postalserver/postal/compare/3.0.2...3.1.0) (2024-03-06)
+
+
+### Features
+
+* configurable trusted proxies for web requests ([3785c99](https://github.com/postalserver/postal/commit/3785c998513c634d225b489ccb43e926ce3f270a))
+
+
+### Bug Fixes
+
+* **message-dequeuer:** ensure SMTP endpoints are sent to SMTP sender appropriately ([e2d642c](https://github.com/postalserver/postal/commit/e2d642c0cbf443550886d90abc3a6edf3e4bc4fc)), closes [#2853](https://github.com/postalserver/postal/issues/2853)
+* **smtp-server:** listen on all interfaces by default ([d1e5b68](https://github.com/postalserver/postal/commit/d1e5b68200ea4b9710cc8714afb3271bad1f4f66)), closes [#2852](https://github.com/postalserver/postal/issues/2852)
+* **smtp-server:** remove ::ffff: from the start of ipv4 addresses ([0dc7359](https://github.com/postalserver/postal/commit/0dc7359431001c9ef1222913f8d1344093397596))
+* **smtp-server:** reset ansi sequence after logging ([9bf6152](https://github.com/postalserver/postal/commit/9bf6152060ffb8b611b66818c1d1ac7c929b7ffe))
+* **ui:** fixes typo on queue page ([33513a7](https://github.com/postalserver/postal/commit/33513a77c0df24d832ab7ed5237d68e2b1bde887))
+* **web-server:** allow for trusted proxies not be set ([4e1deb2](https://github.com/postalserver/postal/commit/4e1deb2d2aeb61d9dddb3729916411c94e73c1c6))
+
+
+### Styles
+
+* **rubocop:** use _ when not using a variable in helm config exporter ([2c20ba6](https://github.com/postalserver/postal/commit/2c20ba65f64ccb0f8174e3f523dedb3806478782))
+
+## [3.0.2](https://github.com/postalserver/postal/compare/3.0.1...3.0.2) (2024-03-05)
+
+
+### Bug Fixes
+
+* default to listening on all addresses when using legacy config ([48f6494](https://github.com/postalserver/postal/commit/48f6494240eb0374d5f865425b362e6f306b2653))
+
+
+### Miscellaneous Chores
+
+* removing arm64 support until we can get a reasonable build pipeline sorted ([e8e44f5](https://github.com/postalserver/postal/commit/e8e44f54b09426c8a04e466bc037851a0833d124))
+
+## [3.0.1](https://github.com/postalserver/postal/compare/3.0.0...3.0.1) (2024-03-05)
+
+
+### Bug Fixes
+
+* fix issue with sending mail when smtp relays are configured ([6dd6e29](https://github.com/postalserver/postal/commit/6dd6e29929c70eaa8b9d3b33c184996b0b6abb82))
+
+## [3.0.0](https://github.com/postalserver/postal/compare/2.3.2...3.0.0) (2024-03-04)
+
+This version of Postal introduces a number of larger changes. Please be sure to follow the [upgrade guide](https://docs.postalserver.io/getting-started/upgrade-to-v3) when upgrading to Postal v3. Highlights include:
+
+* Removal of RabbitMQ dependency
+* Removal of `cron` and `requeuer` processes
+* Improved logging
+* Improved configuration
+* Adds prometheus metric exporters for workers and SMTP servers
+* Only accepted RFC-compliant end-of-DATA sequences (to avoid SMTP smuggling)
+
+### Features
+
+* add health server and prometheus metrics to worker ([a2eb70e](https://github.com/postalserver/postal/commit/a2eb70e))
+* add option to delay starting processes until all migrations are run ([1c5ff5a](https://github.com/postalserver/postal/commit/1c5ff5a))
+* add prometheus metrics to smtp server ([2e7b36c](https://github.com/postalserver/postal/commit/2e7b36c))
+* add prometheus metrics to worker ([bea7450](https://github.com/postalserver/postal/commit/bea7450))
+* more consistent logging ([044058d](https://github.com/postalserver/postal/commit/044058d))
+* new background work process ([dc8e895](https://github.com/postalserver/postal/commit/dc8e895))
+* new configuration system (and schema) (#2819) ([0163ac3](https://github.com/postalserver/postal/commit/0163ac3))
+* only accept RFC-compliant End-of-DATA sequence ([0140dc4](https://github.com/postalserver/postal/commit/0140dc4))
+* add "postal:update" task ([b35eea6](https://github.com/postalserver/postal/commit/b35eea6338f1888bfac2ed377d0a412680483e90))
+* add helm env var config exporter ([8938988](https://github.com/postalserver/postal/commit/893898835dcd9684ae3401549389b173a3feb1fb))
+* include list-unsubscribe-post header in dkim signatures ([e1bae60](https://github.com/postalserver/postal/commit/e1bae60768c4cf151d5a6a141985c78753dce02d)), closes [#2789](https://github.com/postalserver/postal/issues/2789) [#2788](https://github.com/postalserver/postal/issues/2788)
+* support for additional SMTP client options ([0daa667](https://github.com/postalserver/postal/commit/0daa667b55fd9b948da643d37ec438e341809369))
+
+
+### Bug Fixes
+
+* fixes potential issue if machine hostname cannot be determined ([0fcf778](https://github.com/postalserver/postal/commit/0fcf778))
+* raise an error if MX lookup times out during sending ([fadca88](https://github.com/postalserver/postal/commit/fadca88)), closes [#2833](https://github.com/postalserver/postal/issues/2833)
+* set correct component in health server log lines ([a7a9a18](https://github.com/postalserver/postal/commit/a7a9a18))
+* translate unicode domain names to Punycode for compatibility with DNS (#2823) ([be0df7b](https://github.com/postalserver/postal/commit/be0df7b))
+* remove user email verification ([e05f0b3](https://github.com/postalserver/postal/commit/e05f0b3616da8b962b763c48a2139882fd88047a))
+* unescape in URLs which are stored for tracking ([1da1182](https://github.com/postalserver/postal/commit/1da1182c23e9673d8f109d8ed29e80983cdccabf)), closes [#2568](https://github.com/postalserver/postal/issues/2568) [#907](https://github.com/postalserver/postal/issues/907) [#115](https://github.com/postalserver/postal/issues/115)
+* update wording about tracking domain cnames ([0d3eccb](https://github.com/postalserver/postal/commit/0d3eccb368630b4fd21bd858a7829ec00c35f153)), closes [#2808](https://github.com/postalserver/postal/issues/2808)
+
+
+### Documentation
+
+* add message_db.encoding to config docs ([0c1f925](https://github.com/postalserver/postal/commit/0c1f925))
+* add new repo readme welcome image ([afa1726](https://github.com/postalserver/postal/commit/afa1726))
+* add quick contributing instructions ([8d21adc](https://github.com/postalserver/postal/commit/8d21adc))
+* update SECURITY policy ([cfc1c9b](https://github.com/postalserver/postal/commit/cfc1c9b))
+* update docs for how IP address allocation works ([07eb152](https://github.com/postalserver/postal/commit/07eb152)), closes [#2209](https://github.com/postalserver/postal/issues/2209)
+
+### Miscellaneous Chores
+
+* upgrade bundler to 2.5.6 ([1ae8ef6](https://github.com/postalserver/postal/commit/1ae8ef6))
+* upgrade rails to 7.0 and other dependencies ([ecd09a2](https://github.com/postalserver/postal/commit/ecd09a2))
+* upgrade ruby to 3.2.2 and nodejs to 20.x ([72715fe](https://github.com/postalserver/postal/commit/72715fe))
+
+## [2.3.2](https://github.com/postalserver/postal/compare/2.3.1...2.3.2) (2024-03-01)
+
+
+### Bug Fixes
+
+* truncate output and details in deliveries to 250 characters ([694240d](https://github.com/postalserver/postal/commit/694240ddcdef1df9b32888de8fb743d2dee86462)), closes [#2831](https://github.com/postalserver/postal/issues/2831)
+
+## [2.3.1](https://github.com/postalserver/postal/compare/2.3.0...2.3.1) (2024-02-23)
+
+
+### Bug Fixes
+
+* update raw headers after changing messages to during parsing ([2834e2c](https://github.com/postalserver/postal/commit/2834e2c37971db9b0b0498e38b382cf1f8ee26eb)), closes [#2816](https://github.com/postalserver/postal/issues/2816)
+
+
+### Miscellaneous Chores
+
+* **github-actions:** add 'docs' label to exclude from staleness checks ([57b72fb](https://github.com/postalserver/postal/commit/57b72fb4b7f7fc934cfa23906de65b8f6d6d1978))
+* **github-actions:** add action to close stale issues and PRs ([d90a456](https://github.com/postalserver/postal/commit/d90a456dfa661d87e820160d2045c73c765564d2))
+* **github-actions:** allow stale action to be run on demand ([559b08d](https://github.com/postalserver/postal/commit/559b08ddd31ecd904fd09c1e2822161b853166b9))
+
+## [2.3.0](https://github.com/postalserver/postal/compare/2.2.2...2.3.0) (2024-02-13)
+
+
+### Features
+
+* privacy mode ([15f9671](https://github.com/postalserver/postal/commit/15f9671b667cf369255aaa27ee4257267990095c))
+* remove strip_received_headers config option ([ed2e62b](https://github.com/postalserver/postal/commit/ed2e62b94fe76d7aeca0ede98f11a1c4f94c5996))
+
+
+### Bug Fixes
+
+* add ruby platform to gemfile ([1fceef7](https://github.com/postalserver/postal/commit/1fceef7cea76352fd6166fb3f1e8d0ff8591078e))
+* explicitly disable TLS & starttls for unknown SSL modes ([42ab5b3](https://github.com/postalserver/postal/commit/42ab5b3a6b21992c89f8479137699dc9090f0ccc)), closes [#2564](https://github.com/postalserver/postal/issues/2564)
+* fix bug with received header in SMTP server ([ba5bfbd](https://github.com/postalserver/postal/commit/ba5bfbd6a0af9ea33bedb2948822417bd1a3fbc5))
+* retry mysql connections on message DB pool ([f9f7fb3](https://github.com/postalserver/postal/commit/f9f7fb30fee46b661b6dccde4362383ea591532b))
+* use correct method for disconnecting smtp connections ([7c23994](https://github.com/postalserver/postal/commit/7c23994d243ec7d9a17ee053f8c3fa8de0a33097))
+
+
+### Styles
+
+* **rubocop:** disable complexity cops for now ([930cf39](https://github.com/postalserver/postal/commit/930cf39dba37401f90e293c938dee07daf3d9a31))
+* **rubocop:** disable Style/SpecialGlobalVars ([be97f43](https://github.com/postalserver/postal/commit/be97f4330897f96085eb29ed7019b1a3e50af88e))
+* **rubocop:** disable Style/StringConcatenation cop ([d508772](https://github.com/postalserver/postal/commit/d508772a40ef26e5c3a8304aa1f2b8c7985081bd))
+* **rubocop:** Layout/* ([0e0aca0](https://github.com/postalserver/postal/commit/0e0aca06c90f6d2f4db1c4090a35c4537c76e13a))
+* **rubocop:** Layout/EmptyLineAfterMagicComment ([0e4ed5c](https://github.com/postalserver/postal/commit/0e4ed5ca0393f9a56e1efa7ae377d2e4b876bfe1))
+* **rubocop:** Layout/EmptyLines ([0cf35a8](https://github.com/postalserver/postal/commit/0cf35a83926d499a279775bcc32dd4ea79b7a8c9))
+* **rubocop:** Layout/EmptyLinesAroundBlockBody ([cfd8d63](https://github.com/postalserver/postal/commit/cfd8d63321d1821aad7fa9d6b8462c3d551aca61))
+* **rubocop:** Layout/LeadingCommentSpace ([59f299b](https://github.com/postalserver/postal/commit/59f299b704533488b74075beb8692397eb434aab))
+* **rubocop:** Layout/LineLength ([e142d0d](https://github.com/postalserver/postal/commit/e142d0da5fbee19e6f9f1741ff9dee0a2d7dd169))
+* **rubocop:** Layout/MultilineMethodCallBraceLayout ([a5d5ba5](https://github.com/postalserver/postal/commit/a5d5ba5326728413bb95456e92c854977d225a7f))
+* **rubocop:** Lint/DuplicateBranch ([a1dc0f7](https://github.com/postalserver/postal/commit/a1dc0f77ac69937d7f30c9401608dfbe66987d45))
+* **rubocop:** Lint/DuplicateMethods ([bab6346](https://github.com/postalserver/postal/commit/bab6346239e4f50bdd51101c45f3a0cd66f47096))
+* **rubocop:** Lint/IneffectiveAccessModifier ([6ad56ee](https://github.com/postalserver/postal/commit/6ad56ee9c9e5bad19b065fcec3ada3280adbb1f4))
+* **rubocop:** Lint/MissingSuper ([4674e63](https://github.com/postalserver/postal/commit/4674e63b5ff84307f5b772e870e88109af2daf52))
+* **rubocop:** Lint/RedundantStringCoercion ([12a5ef3](https://github.com/postalserver/postal/commit/12a5ef3279bf6c1e5c38bf7e846de1d17bf98c09))
+* **rubocop:** Lint/ShadowedException ([0966b44](https://github.com/postalserver/postal/commit/0966b44018bc1e2f131358635776fcc3b75ee8eb))
+* **rubocop:** Lint/ShadowingOuterLocalVariable ([7119e86](https://github.com/postalserver/postal/commit/7119e8642dffeee7a27f90145073e45664ea58d3))
+* **rubocop:** Lint/SuppressedException ([278ef08](https://github.com/postalserver/postal/commit/278ef0886ac53e6bed15793301dc69c95a37dbde))
+* **rubocop:** Lint/UnderscorePrefixedVariableName ([ec7dcf4](https://github.com/postalserver/postal/commit/ec7dcf4f9a0bdb367a90f1a3b35336909ebc60d7))
+* **rubocop:** Lint/UnusedBlockArgument ([ee94e4e](https://github.com/postalserver/postal/commit/ee94e4e1a013bbe8fbdd8ef94f15ed0fa20709ac))
+* **rubocop:** Lint/UselessAssignment ([7590a46](https://github.com/postalserver/postal/commit/7590a462341bddd412e660db9546ba1909aea9d7))
+* **rubocop:** Naming/FileName ([919a601](https://github.com/postalserver/postal/commit/919a60116c5d81ed787061ff4614da4f1e067d4e))
+* **rubocop:** Naming/MemoizedInstanceVariableName ([9563f30](https://github.com/postalserver/postal/commit/9563f30c96fba12073e845319b8d79a542d88109))
+* **rubocop:** relax method length and block nexting for now ([b0ac9ef](https://github.com/postalserver/postal/commit/b0ac9ef0b96ab78c2961f45b6e9f20f87a6f1d07))
+* **rubocop:** remaining offences ([ec63666](https://github.com/postalserver/postal/commit/ec636661d5c4b9e8f48e6f263ffef834acb68b39))
+* **rubocop:** Security/YAMLLoad ([389ea77](https://github.com/postalserver/postal/commit/389ea7705047bf8700836137514b2497af3c6c01))
+* **rubocop:** Style/AndOr ([b9f3f31](https://github.com/postalserver/postal/commit/b9f3f313f8ec992917bad3a51f0481f89675e935))
+* **rubocop:** Style/ClassAndModuleChildren ([e896f46](https://github.com/postalserver/postal/commit/e896f4689a8fc54979f0a6c2b7ce14746856bad6))
+* **rubocop:** Style/For ([04a3483](https://github.com/postalserver/postal/commit/04a34831c74a3a44547f93100c35db650bc4eef6))
+* **rubocop:** Style/FrozenStringLiteralComment ([6ab36c0](https://github.com/postalserver/postal/commit/6ab36c09c966eb9a8b8ada52155f74d2537977f2))
+* **rubocop:** Style/GlobalStdStream ([75be690](https://github.com/postalserver/postal/commit/75be6907483ea25f828461eb790d3f6f46ca683b))
+* **rubocop:** Style/GlobalVars ([157d114](https://github.com/postalserver/postal/commit/157d11457c520147807901b75b3ba22d29172f24))
+* **rubocop:** Style/HashEachMethods ([c995027](https://github.com/postalserver/postal/commit/c995027ff53962ae49341372f75e2bf43ecde0d2))
+* **rubocop:** Style/HashExcept ([83ac764](https://github.com/postalserver/postal/commit/83ac76451071f097e7197f77fc5ad16e9cf58593))
+* **rubocop:** Style/IdenticalConditionalBranches ([6a58ecf](https://github.com/postalserver/postal/commit/6a58ecf605250b8fa891cb14fca0c0e0ce0d7eb9))
+* **rubocop:** Style/MissingRespondToMissing ([ffcb707](https://github.com/postalserver/postal/commit/ffcb707247fe2a69905aa6e4dc668abeb9924611))
+* **rubocop:** Style/MultilineBlockChain ([c6326a6](https://github.com/postalserver/postal/commit/c6326a6524e6d71d23bc2c256f3f9416c38b0846))
+* **rubocop:** Style/MutableConstant ([129dffa](https://github.com/postalserver/postal/commit/129dffab9ed8726ca4066e7052adc699129de2d2))
+* **rubocop:** Style/NumericPredicate ([c558f1c](https://github.com/postalserver/postal/commit/c558f1c69ce9498564161d8cef3fcb8213103498))
+* **rubocop:** Style/PreferredHashMethods ([013b3ea](https://github.com/postalserver/postal/commit/013b3ea9315c14f24b08574d68e1688f33d78b8d))
+* **rubocop:** Style/SafeNavigation ([00a02f2](https://github.com/postalserver/postal/commit/00a02f2655b6e3296ad0e7ea9b9872da936b56ed))
+* **rubocop:** Style/SelectByRegexp ([9ce28a4](https://github.com/postalserver/postal/commit/9ce28a427fadf6fafd942e009792be4b5539d40d))
+* **rubocop:** Style/StringLiterals ([b4cc812](https://github.com/postalserver/postal/commit/b4cc81264c85d5f0061e5b5121a30d7bdcabc189))
+* **rubocop:** Style/WordArray ([bd85920](https://github.com/postalserver/postal/commit/bd8592056573d1c6933d753ed50bdf9a7466e761))
+* **rubocop:** update rubocop rules ([6d4dea7](https://github.com/postalserver/postal/commit/6d4dea7f7f0145ff2b99cf2505bc70a3e5925256))
+
+
+### Miscellaneous Chores
+
+* annotate models ([6214892](https://github.com/postalserver/postal/commit/6214892710e21c2aa29a319d5809f7bdf0d50529))
+* silence message DB migrations during provisioning ([c83601a](https://github.com/postalserver/postal/commit/c83601af69f35e338b1f7c10ef7994f74b96f8bf))
+
+
+### Code Refactoring
+
+* remove reloading on the smtp server ([c3c304e](https://github.com/postalserver/postal/commit/c3c304e98b3274433248792b6403acf63d7a513b))
+
+
+### Tests
+
+* add initial tests for Postal::SMTPServer::Client ([dece1d4](https://github.com/postalserver/postal/commit/dece1d487ac2fdce104700939a79a5579b60a0cb))
+* FactoryBot.lint will lint all registered factories ([25d7d66](https://github.com/postalserver/postal/commit/25d7d66b4709fe5442d554097a6ef074aeb15f72))
+* remove FACTORIES_EXCLUDED_FROM_LINT ([1cf665a](https://github.com/postalserver/postal/commit/1cf665a0cf61d1eae3d08bdadf6fccaab6413023))
+
+## [2.2.2](https://github.com/postalserver/postal/compare/2.2.1...2.2.2) (2024-02-06)
+
+
+### Bug Fixes
+
+* adds new connection pool which will discard failed clients ([54306a9](https://github.com/postalserver/postal/commit/54306a974802c2e4d17e0980531e2d0dba08150a)), closes [#2780](https://github.com/postalserver/postal/issues/2780)
+* re-add reconnect: true to database ([7bc5230](https://github.com/postalserver/postal/commit/7bc5230cbaae58fb6f8512d1d1b0e6a2eb989b56))
+* upgrade nokogiri ([f05c2e4](https://github.com/postalserver/postal/commit/f05c2e4503688e59a5ef513a5a1064d0ebbb5813))
+
+
+### Tests
+
+* rename database spec file ([b9edcf5](https://github.com/postalserver/postal/commit/b9edcf5b7dda7f4976a9d3f90668bbdacea57350))
+
+## [2.2.1](https://github.com/postalserver/postal/compare/2.2.0...2.2.1) (2024-02-03)
+
+
+### Bug Fixes
+
+* fixes issue starting application in production mode ([4528a14](https://github.com/postalserver/postal/commit/4528a14d273c141e5719f19c3b08c00364b47638))
+
+
+### Code Refactoring
+
+* remove Postal.database_url ([96ba4b8](https://github.com/postalserver/postal/commit/96ba4b8f309cfcd1d605e5c7fc05507b21c78c6f))
+
+## [2.2.0](https://github.com/postalserver/postal/compare/2.1.6...2.2.0) (2024-02-01)
+
+
+### Features
+
+* load signing key path from POSTAL_SIGNING_KEY_PATH ([4a46f69](https://github.com/postalserver/postal/commit/4a46f690de3010f1ae4d6c17739530a4eae35c09))
+* support for configuring postal with environment variables ([854aa5e](https://github.com/postalserver/postal/commit/854aa5ebc87de692b4691d48759aefd6fae9d133))
+
+
+### Bug Fixes
+
+* don't use indifferent access for job params ([2bad645](https://github.com/postalserver/postal/commit/2bad645d980ad4b712a3c863b5350e4ee2895071)), closes [#2477](https://github.com/postalserver/postal/issues/2477) [#2714](https://github.com/postalserver/postal/issues/2714) [#2476](https://github.com/postalserver/postal/issues/2476) [#2500](https://github.com/postalserver/postal/issues/2500)
+* extract x-postal-tag before holding ([6b2bf90](https://github.com/postalserver/postal/commit/6b2bf9062d662ede14617c4995ffaacca023a3b1)), closes [#2684](https://github.com/postalserver/postal/issues/2684)
+* fixes error messages in web ui ([71f51db](https://github.com/postalserver/postal/commit/71f51db3c2515addaf8b280667555427d64796be))
+* ignore message DB migrations in autoloader ([3fb40e4](https://github.com/postalserver/postal/commit/3fb40e4e247893b314e42affa4604a7a71a52c59))
+* move tracking middleware before host authorization ([49cceaa](https://github.com/postalserver/postal/commit/49cceaa6ca862965448041279fc439ecba163ff8)), closes [#2415](https://github.com/postalserver/postal/issues/2415)
+* use utc timestamps when determining raw table names ([ce19bf7](https://github.com/postalserver/postal/commit/ce19bf7988d522bf46aabf68090751427e286ffc))
+
+
+### Miscellaneous Chores
+
+* add binstubs for bundle and rspec ([41f6cf4](https://github.com/postalserver/postal/commit/41f6cf4d909518526af55ecb3fcccfa8fb8e1da2))
+* add script to send html emails to a local SMTP server ([8794a2f](https://github.com/postalserver/postal/commit/8794a2f44783658a075a6f3985079ae4743412b1))
+
+
+### Code Refactoring
+
+* remove explicit autoload ([0f9882f](https://github.com/postalserver/postal/commit/0f9882f13204124df630606b1b9e36787c9c4011))
+* remove Postal::Job.perform method ([990b575](https://github.com/postalserver/postal/commit/990b575902c45bb1678cc95f53ef3166c4b7092e))
+
+## [2.1.6](https://github.com/postalserver/postal/compare/2.1.5...2.1.6) (2024-01-30)
+
+
+### Miscellaneous Chores
+
+* **build:** fixes docker login action credentials ([8810856](https://github.com/postalserver/postal/commit/88108566f8ab33f1a4263a36a5c1ffc071645ac3))
+* update release please to include more categories in changelog ([e156c21](https://github.com/postalserver/postal/commit/e156c21dee304de7d10c2958c493cce73c2d8fea))
+
+## [2.1.5](https://github.com/postalserver/postal/compare/2.1.4...2.1.5) (2024-01-30)
+
+
+### Bug Fixes
+
+* duplicate string before modifying it to prevent frozen string errors ([f0a8aca](https://github.com/postalserver/postal/commit/f0a8aca6e10064fb16daefff9e22dcc20a831868))
+* fixed typo (rfc number) ([2f62baa](https://github.com/postalserver/postal/commit/2f62baa238fc1102706ee4acf079b7a876b05283))
+* fixes typo in on track domains page ([77bd77b](https://github.com/postalserver/postal/commit/77bd77b629fcbc44b8d27deb0d33a457b02309f2))
+* mail view encoding issue [#2462](https://github.com/postalserver/postal/issues/2462) ([#2596](https://github.com/postalserver/postal/issues/2596)) ([59f4478](https://github.com/postalserver/postal/commit/59f44781973489817efb5b3435d95d25f44f90ce))
+* match IPv4 mapped IPv6 addresses when searching for SMTP-IP credentials ([8b525d0](https://github.com/postalserver/postal/commit/8b525d0381a9e0113af808b9ec2eb47bf78ec60b))
+
+## 2.1.4
+
+### Bug Fixes
+
+- Move RubyVer functionality to Utilities module ([5998bf](https://github.com/postalserver/postal/commit/5998bf376a274df19f29877e7f68ea75f298c9f9))
+
+## 2.1.3
+
+### Features
+- Upgrade to Ruby 3.2.1 & Rails 6.1 ([957b78](https://github.com/postalserver/postal/commit/957b784658cda8c4c95cf1f2b65e05d99d23d427))
+- Make resent-sender header optional ([c6fb8d](https://github.com/postalserver/postal/commit/c6fb8d223bdeaccdc9e8bdbd031fe3f325ac0677))
+- Log CRAM-MD5 authentication failures ([9b1ed1](https://github.com/postalserver/postal/commit/9b1ed1e7e16a8f55a5bd7b7ce72195a08ca2968d))
+- Always use multipart/alternative parts in generated emails ([d0db13](https://github.com/postalserver/postal/commit/d0db1345a2bf8f538b01b974e74391da6fffe2b1))
+
+### Bug Fixes
+
+- Use non-blocking function to negotiate TLS connections ([a7dd19](https://github.com/postalserver/postal/commit/a7dd19baac8300f4d8ee89d0050479e08fdf9176))
+- Fix to newline conversion process ([9f4ef8](https://github.com/postalserver/postal/commit/9f4ef8f57a839c5529b4f00a36b832740386b4ed))
+- Remove custom scrollbars ([b22f1b](https://github.com/postalserver/postal/commit/b22f1bdb2e2d66b096ca993d6a5f4f708274a4a2))
+- Truncate 'output' field to avoid overflowing varchar(512) in database ([a188a1](https://github.com/postalserver/postal/commit/a188a161cbdcfd70158b09b53cef622842357c26))
+- Fix link replacement in multipart messsages ([7ea00d](https://github.com/postalserver/postal/commit/7ea00dfa3bc3c7650cc2b134beacbff22101a913))
+- Fix confusing error message when deleting IP pools ([cefc7d](https://github.com/postalserver/postal/commit/cefc7d17b82f610001859a8e323ee1dfde149ba5))
+- Connect to correct IP rather than hostname suring SMTP delivery ([159509](https://github.com/postalserver/postal/commit/159509a3ed29ae33cba522b255904992922dcfdf))
+- Change retry timings to avoid re-sending messages too early ([c8d27b](https://github.com/postalserver/postal/commit/c8d27b2963af122d6555abdf0742d2d2d6f11ce5))
+
 ## 2.1.2
 
 ### Features

CONTRIBUTING.md

@@ -0,0 +1,53 @@
+# Contributing to Postal
+
+This doc explains how to go about running Postal in development to allow you to make contributions to the project.
+
+## Dependencies
+
+You will need a MySQL database server to get started. Postal needs to be able to make databases within that server whenever new mail servers are created so the permissions that you use should be suitable for that.
+
+You'll also need Ruby. Postal currently uses the Ruby version in `.ruby-version`. Install that using whichever version manager takes your fancy - rbenv, asdf, rvm etc.
+
+## Clone
+
+You'll need to clone the repository
+
+```
+git clone git@github.com:postalserver/postal
+```
+
+Once cloned, you can install the Ruby dependencies using bundler.
+
+```
+bundle install
+```
+
+## Configuration
+
+Configuration is handled using a config file. This lives in `config/postal/postal.yml`. An example configuration file is provided in `config/examples/development.yml`. This example is for development use only and not an example for production use.
+
+You'll also need a key for signing. You can generate one of these like this:
+
+```
+openssl genrsa -out config/postal/signing.key 2048
+```
+
+If you're running the tests (and you probably should be), you'll find an example file for test configuration in `config/examples/test.yml`. This should be placed in `config/postal/postal.test.yml` with the appropriate values.
+
+If you prefer, you can configure Postal using environment variables. These should be placed in `.env` or `.env.test` as apprpriate.
+
+## Running
+
+The neatest way to run postal is to ensure that `./bin` is your `$PATH` and then use one of the following commands.
+
+* `bin/dev` - will run all components of the application using Foreman
+* `bin/postal` - will run the Postal binary providing access to running individual components or other tools.
+
+## Database initialization
+
+Use the commands below to initialize your database and make your first user.
+
+```
+postal initialize
+postal make-user
+```

Dockerfile

@@ -1,25 +1,25 @@
-FROM ruby:2.6-buster AS base
+FROM ruby:3.4.10-slim-bookworm AS base
 
 SHELL ["/bin/bash", "-o", "pipefail", "-c"]
 RUN apt-get update \
-  && apt-get install -y --no-install-recommends \
-  software-properties-common dirmngr apt-transport-https \
-  && apt-key adv --fetch-keys 'https://mariadb.org/mariadb_release_signing_key.asc' \
-  && add-apt-repository 'deb [arch=amd64,arm64,ppc64el] https://mirrors.xtom.nl/mariadb/repo/10.6/debian buster main' \
-  && (curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -) \
-  && (echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list) \
-  && (curl -sL https://deb.nodesource.com/setup_12.x | bash -) \
+  && apt-get install --no-install-recommends -y curl \
+  && apt-get clean \
   && rm -rf /var/lib/apt/lists/*
 
+RUN (curl -sL https://deb.nodesource.com/setup_20.x | bash -)
+
 # Install main dependencies
 RUN apt-get update && \
   apt-get install -y --no-install-recommends \
-  build-essential  \
-  netcat \
-  curl \
-  libmariadbclient-dev \
-  nano \
-  nodejs
+    build-essential  \
+    netcat-openbsd \
+    libmariadb-dev \
+    libcap2-bin \
+    nano \
+    libyaml-dev \
+    nodejs \
+  && apt-get clean \
+  && rm -rf /var/lib/apt/lists/*
 
 RUN setcap 'cap_net_bind_service=+ep' /usr/local/bin/ruby
 
@@ -34,23 +34,25 @@ RUN mkdir -p /opt/postal/app /opt/postal/config
 WORKDIR /opt/postal/app
 
 # Install bundler
-RUN gem install bundler -v 2.1.4 --no-doc
+RUN gem install bundler -v 2.7.2 --no-doc
 
 # Install the latest and active gem dependencies and re-run
 # the appropriate commands to handle installs.
-COPY Gemfile Gemfile.lock ./
-RUN bundle install -j 4
+COPY --chown=postal Gemfile Gemfile.lock ./
+RUN bundle install
 
 # Copy the application (and set permissions)
 COPY ./docker/wait-for.sh /docker-entrypoint.sh
 COPY --chown=postal . .
 
 # Export the version
-ARG VERSION=unspecified
-RUN echo $VERSION > VERSION
+ARG VERSION
+ARG BRANCH
+RUN if [ "$VERSION" != "" ]; then echo $VERSION > VERSION; fi \
+  && if [ "$BRANCH" != "" ]; then echo $BRANCH > BRANCH; fi
 
-# Set the path to the config
-ENV POSTAL_CONFIG_ROOT=/config
+# Set paths for when running in a container
+ENV POSTAL_CONFIG_FILE_PATH=/config/postal.yml
 
 # Set the CMD
 ENTRYPOINT [ "/docker-entrypoint.sh" ]
@@ -59,8 +61,8 @@ CMD ["postal"]
 # ci target - use --target=ci to skip asset compilation
 FROM base AS ci
 
-# prod target - default if no --target option is given
-FROM base AS prod
+# full target - default if no --target option is given
+FROM base AS full
 
-RUN POSTAL_SKIP_CONFIG_CHECK=1 RAILS_GROUPS=assets bundle exec rake assets:precompile
+RUN RAILS_GROUPS=assets bundle exec rake assets:precompile
 RUN touch /opt/postal/app/public/assets/.prebuilt

Gemfile

@@ -1,50 +1,72 @@
-source 'https://rubygems.org'
-gem 'rails', '= 5.2.6.2'
-gem 'mysql2'
-gem 'puma'
-gem 'turbolinks', '~> 5'
-gem 'haml'
-gem 'nifty-utils'
-gem 'nilify_blanks'
-gem 'kaminari'
-gem 'bcrypt'
-gem 'foreman'
-gem 'hashie'
-gem 'authie', '~> 3.0'
-gem 'dynamic_form'
-gem 'changey'
-gem 'mail', :git => 'https://github.com/mikel/mail.git', :branch => '2-7-stable'
-gem 'autoprefixer-rails'
-gem 'bunny'
-gem 'secure_headers'
-gem 'chronic'
-gem 'basic_ssl'
-gem 'clockwork'
-gem 'encrypto_signo'
-gem 'nio4r'
-gem 'sentry-raven'
-gem 'gelf'
-gem 'moonrope'
-gem 'jwt'
-gem 'highline', :require => false
-gem 'resolv', '~> 0.2.1'
-gem 'dotenv-rails'
+# frozen_string_literal: true
 
-group :development, :assets do
-  gem 'sass-rails', '~> 5.0'
-  gem 'uglifier', '>= 1.3.0'
-  gem 'coffee-rails', '~> 4.2'
-  gem 'jquery-rails'
+source "https://rubygems.org"
+gem "abbrev"
+gem "authie"
+gem "autoprefixer-rails"
+gem "bcrypt"
+gem "chronic"
+gem "domain_name"
+gem "dotenv"
+gem "execjs"
+gem "gelf"
+gem "haml"
+gem "hashie"
+gem "highline", require: false
+# json 3.0 removed the `quirks_mode` option, which Active Support 7.2 still
+# passes in both its JSON encoder and decoder. Nothing else constrains json, so
+# without this pin Bundler resolves to 3.x and every `to_json` call raises
+# ArgumentError. Active Support dropped those calls in Rails 8.1, so this pin
+# can go when we move to 8.1.
+gem "json", "< 3"
+gem "jwt"
+gem "kaminari"
+gem "klogger-logger"
+gem "konfig-config", "~> 3.0"
+gem "logger"
+gem "mail"
+gem "mutex_m"
+gem "mysql2"
+gem "nifty-utils"
+gem "nilify_blanks"
+gem "nio4r"
+gem "ostruct"
+gem "prometheus-client"
+gem "puma"
+gem "rackup"
+gem "rails", "= 7.2.3.2"
+gem "resolv"
+gem "secure_headers"
+gem "sentry-rails"
+gem "turbolinks", "~> 5"
+gem "webrick"
+
+group :oidc do
+  # These are gems which are needed for OpenID connect. They are only required by the application
+  # when OIDC is enabled in the Postal configuration.
+  gem "omniauth_openid_connect"
+  gem "omniauth-rails_csrf_protection"
 end
 
-group :development, :test do
-  gem 'byebug'
+group :development, :test, :assets do
+  gem "coffee-rails", "~> 5.0"
+  gem "jquery-rails"
+  gem "sass-rails"
+  gem "uglifier", ">= 1.3.0"
 end
 
 group :development do
-  gem 'annotate'
-  gem 'rspec'
-  gem 'rspec-rails'
-  gem "factory_bot_rails", "~> 4.0"
-  gem "database_cleaner"
+  gem "annotate"
+  gem "rubocop"
+  gem "rubocop-rails"
+end
+
+group :test do
+  gem "database_cleaner-active_record"
+  gem "factory_bot_rails"
+  gem "rspec"
+  gem "rspec-rails"
+  gem "shoulda-matchers"
+  gem "timecop"
+  gem "webmock"
 end

Gemfile.lock

@@ -1,293 +1,511 @@
-GIT
-  remote: https://github.com/mikel/mail.git
-  revision: 873aa3876efb372b9a0bb7319fe60a994cedfd05
-  branch: 2-7-stable
-  specs:
-    mail (2.7.2.edge)
-      mini_mime (>= 0.1.1)
-
 GEM
   remote: https://rubygems.org/
   specs:
-    actioncable (5.2.6.2)
-      actionpack (= 5.2.6.2)
+    abbrev (0.1.2)
+    actioncable (7.2.3.2)
+      actionpack (= 7.2.3.2)
+      activesupport (= 7.2.3.2)
       nio4r (~> 2.0)
       websocket-driver (>= 0.6.1)
-    actionmailer (5.2.6.2)
-      actionpack (= 5.2.6.2)
-      actionview (= 5.2.6.2)
-      activejob (= 5.2.6.2)
-      mail (~> 2.5, >= 2.5.4)
-      rails-dom-testing (~> 2.0)
-    actionpack (5.2.6.2)
-      actionview (= 5.2.6.2)
-      activesupport (= 5.2.6.2)
-      rack (~> 2.0, >= 2.0.8)
+      zeitwerk (~> 2.6)
+    actionmailbox (7.2.3.2)
+      actionpack (= 7.2.3.2)
+      activejob (= 7.2.3.2)
+      activerecord (= 7.2.3.2)
+      activestorage (= 7.2.3.2)
+      activesupport (= 7.2.3.2)
+      mail (>= 2.8.0)
+    actionmailer (7.2.3.2)
+      actionpack (= 7.2.3.2)
+      actionview (= 7.2.3.2)
+      activejob (= 7.2.3.2)
+      activesupport (= 7.2.3.2)
+      mail (>= 2.8.0)
+      rails-dom-testing (~> 2.2)
+    actionpack (7.2.3.2)
+      actionview (= 7.2.3.2)
+      activesupport (= 7.2.3.2)
+      cgi
+      nokogiri (>= 1.8.5)
+      racc
+      rack (>= 2.2.4, < 3.3)
+      rack-session (>= 1.0.1)
       rack-test (>= 0.6.3)
-      rails-dom-testing (~> 2.0)
-      rails-html-sanitizer (~> 1.0, >= 1.0.2)
-    actionview (5.2.6.2)
-      activesupport (= 5.2.6.2)
+      rails-dom-testing (~> 2.2)
+      rails-html-sanitizer (~> 1.6)
+      useragent (~> 0.16)
+    actiontext (7.2.3.2)
+      actionpack (= 7.2.3.2)
+      activerecord (= 7.2.3.2)
+      activestorage (= 7.2.3.2)
+      activesupport (= 7.2.3.2)
+      globalid (>= 0.6.0)
+      nokogiri (>= 1.8.5)
+    actionview (7.2.3.2)
+      activesupport (= 7.2.3.2)
       builder (~> 3.1)
-      erubi (~> 1.4)
-      rails-dom-testing (~> 2.0)
-      rails-html-sanitizer (~> 1.0, >= 1.0.3)
-    activejob (5.2.6.2)
-      activesupport (= 5.2.6.2)
+      cgi
+      erubi (~> 1.11)
+      rails-dom-testing (~> 2.2)
+      rails-html-sanitizer (~> 1.6)
+    activejob (7.2.3.2)
+      activesupport (= 7.2.3.2)
       globalid (>= 0.3.6)
-    activemodel (5.2.6.2)
-      activesupport (= 5.2.6.2)
-    activerecord (5.2.6.2)
-      activemodel (= 5.2.6.2)
-      activesupport (= 5.2.6.2)
-      arel (>= 9.0)
-    activestorage (5.2.6.2)
-      actionpack (= 5.2.6.2)
-      activerecord (= 5.2.6.2)
-      marcel (~> 1.0.0)
-    activesupport (5.2.6.2)
-      concurrent-ruby (~> 1.0, >= 1.0.2)
-      i18n (>= 0.7, < 2)
-      minitest (~> 5.1)
-      tzinfo (~> 1.1)
-    amq-protocol (2.3.0)
-    annotate (2.7.4)
-      activerecord (>= 3.2, < 6.0)
-      rake (>= 10.4, < 13.0)
-    arel (9.0.0)
-    authie (3.3.1)
-      secure_random_string
-    autoprefixer-rails (9.1.3)
-      execjs
-    basic_ssl (1.0.3)
-    bcrypt (3.1.12)
-    builder (3.2.4)
-    bunny (2.14.4)
-      amq-protocol (~> 2.3, >= 2.3.0)
-    byebug (10.0.2)
-    changey (1.0.0)
-      activerecord (>= 4.2, < 6)
+    activemodel (7.2.3.2)
+      activesupport (= 7.2.3.2)
+    activerecord (7.2.3.2)
+      activemodel (= 7.2.3.2)
+      activesupport (= 7.2.3.2)
+      timeout (>= 0.4.0)
+    activestorage (7.2.3.2)
+      actionpack (= 7.2.3.2)
+      activejob (= 7.2.3.2)
+      activerecord (= 7.2.3.2)
+      activesupport (= 7.2.3.2)
+      marcel (~> 1.0)
+    activesupport (7.2.3.2)
+      base64
+      benchmark (>= 0.3)
+      bigdecimal
+      concurrent-ruby (~> 1.0, >= 1.3.1)
+      connection_pool (>= 2.2.5)
+      drb
+      i18n (>= 1.6, < 2)
+      logger (>= 1.4.2)
+      minitest (>= 5.1, < 6)
+      securerandom (>= 0.3)
+      tzinfo (~> 2.0, >= 2.0.5)
+    addressable (2.9.0)
+      public_suffix (>= 2.0.2, < 8.0)
+    aes_key_wrap (1.1.0)
+    annotate (3.2.0)
+      activerecord (>= 3.2, < 8.0)
+      rake (>= 10.4, < 14.0)
+    ast (2.4.3)
+    attr_required (1.0.2)
+    authie (5.0.0)
+      activerecord (>= 6.1, < 9.0)
+    autoprefixer-rails (10.4.21.0)
+      execjs (~> 2)
+    base64 (0.3.0)
+    bcrypt (3.1.22)
+    benchmark (0.5.0)
+    bigdecimal (4.1.3)
+    bindata (3.0.0)
+    builder (3.3.0)
+    cgi (0.5.2)
     chronic (0.10.2)
-    clockwork (2.0.3)
-      tzinfo
-    coffee-rails (4.2.2)
+    coffee-rails (5.0.0)
       coffee-script (>= 2.2.0)
-      railties (>= 4.0.0)
+      railties (>= 5.2.0)
     coffee-script (2.4.1)
       coffee-script-source
       execjs
     coffee-script-source (1.12.2)
-    concurrent-ruby (1.1.9)
-    crass (1.0.6)
-    database_cleaner (1.7.0)
-    deep_merge (1.2.1)
-    diff-lcs (1.3)
-    dotenv (2.7.6)
-    dotenv-rails (2.7.6)
-      dotenv (= 2.7.6)
-      railties (>= 3.2)
-    dynamic_form (1.1.4)
-    encrypto_signo (1.0.0)
-    erubi (1.10.0)
-    execjs (2.7.0)
-    factory_bot (4.11.1)
-      activesupport (>= 3.0.0)
-    factory_bot_rails (4.11.1)
-      factory_bot (~> 4.11.1)
-      railties (>= 3.0.0)
-    faraday (0.15.2)
-      multipart-post (>= 1.2, < 3)
-    ffi (1.15.5)
-    foreman (0.85.0)
-      thor (~> 0.19.1)
-    gelf (3.0.0)
+    concurrent-ruby (1.3.8)
+    connection_pool (3.0.2)
+    crack (1.0.1)
+      bigdecimal
+      rexml
+    crass (1.0.7)
+    database_cleaner-active_record (2.2.2)
+      activerecord (>= 5.a)
+      database_cleaner-core (~> 2.0)
+    database_cleaner-core (2.1.0)
+    date (3.5.1)
+    diff-lcs (1.6.2)
+    domain_name (0.6.20260907)
+    dotenv (3.2.0)
+    drb (2.2.3)
+    erb (6.0.7)
+    erubi (1.13.1)
+    execjs (2.10.2)
+      json (>= 2)
+    factory_bot (6.6.0)
+      activesupport (>= 6.1.0)
+    factory_bot_rails (6.5.1)
+      factory_bot (~> 6.5)
+      railties (>= 6.1.0)
+    faraday (2.14.3)
+      faraday-net_http (>= 2.0, < 3.5)
       json
-    globalid (1.0.0)
-      activesupport (>= 5.0)
-    haml (5.1.2)
-      temple (>= 0.8.0)
+      logger
+    faraday-follow_redirects (0.5.0)
+      faraday (>= 1, < 3)
+    faraday-net_http (3.4.4)
+      net-http (~> 0.5)
+    ffi (1.15.5)
+    gelf (3.1.0)
+      json
+    globalid (1.4.0)
+      activesupport (>= 6.1)
+    haml (7.5.1)
+      prism (>= 1.1.0)
+      temple (>= 0.8.2)
+      thor
       tilt
-    hashie (3.6.0)
-    highline (2.0.0)
-    i18n (1.9.1)
+    hashdiff (1.2.1)
+    hashie (5.1.0)
+      logger
+    highline (3.1.2)
+      reline
+    i18n (1.15.2)
       concurrent-ruby (~> 1.0)
-    jquery-rails (4.3.3)
+    io-console (0.9.2)
+    irb (1.18.0)
+      pp (>= 0.6.0)
+      prism (>= 1.3.0)
+      rdoc (>= 4.0.0)
+      reline (>= 0.4.2)
+    jquery-rails (4.6.1)
       rails-dom-testing (>= 1, < 3)
       railties (>= 4.2.0)
       thor (>= 0.14, < 2.0)
-    json (2.5.1)
-    jwt (2.1.0)
-    kaminari (1.2.1)
+    json (2.21.2)
+    json-jwt (1.17.2)
+      activesupport (>= 4.2)
+      aes_key_wrap
+      base64
+      bindata
+      faraday (~> 2.0)
+      faraday-follow_redirects
+    jwt (3.3.0)
+      base64
+    kaminari (1.2.2)
       activesupport (>= 4.1.0)
-      kaminari-actionview (= 1.2.1)
-      kaminari-activerecord (= 1.2.1)
-      kaminari-core (= 1.2.1)
-    kaminari-actionview (1.2.1)
+      kaminari-actionview (= 1.2.2)
+      kaminari-activerecord (= 1.2.2)
+      kaminari-core (= 1.2.2)
+    kaminari-actionview (1.2.2)
       actionview
-      kaminari-core (= 1.2.1)
-    kaminari-activerecord (1.2.1)
+      kaminari-core (= 1.2.2)
+    kaminari-activerecord (1.2.2)
       activerecord
-      kaminari-core (= 1.2.1)
-    kaminari-core (1.2.1)
-    loofah (2.14.0)
-      crass (~> 1.0.2)
-      nokogiri (>= 1.5.9)
-    marcel (1.0.2)
-    method_source (1.0.0)
-    mini_mime (1.1.2)
-    mini_portile2 (2.7.1)
-    minitest (5.15.0)
-    moonrope (2.0.2)
-      deep_merge (~> 1.0)
+      kaminari-core (= 1.2.2)
+    kaminari-core (1.2.2)
+    klogger-logger (1.5.0)
+      concurrent-ruby (>= 1.0, < 2.0)
       json
-      rack (>= 1.4)
-    multipart-post (2.0.0)
-    mysql2 (0.5.3)
+      rouge (>= 3.30, < 5.0)
+    konfig-config (3.0.0)
+      hashie
+    language_server-protocol (3.17.0.6)
+    lint_roller (1.1.0)
+    logger (1.7.0)
+    loofah (2.25.2)
+      crass (~> 1.0.2)
+      nokogiri (>= 1.12.0)
+    mail (2.9.1)
+      logger
+      mini_mime (>= 0.1.1)
+      net-imap
+      net-pop
+      net-smtp
+    marcel (1.2.1)
+    mini_mime (1.1.5)
+    minitest (5.27.0)
+    mutex_m (0.3.0)
+    mysql2 (0.5.7)
+      bigdecimal
+    net-http (0.9.1)
+      uri (>= 0.11.1)
+    net-imap (0.6.7)
+      date
+      net-protocol
+    net-pop (0.1.2)
+      net-protocol
+    net-protocol (0.3.0)
+      timeout
+    net-smtp (0.5.1)
+      net-protocol
     nifty-utils (1.1.7)
-    nilify_blanks (1.3.0)
-      activerecord (>= 3.0.0)
-      activesupport (>= 3.0.0)
-    nio4r (2.5.8)
-    nokogiri (1.13.1)
-      mini_portile2 (~> 2.7.0)
+    nilify_blanks (1.4.0)
+      activerecord (>= 4.0.0)
+      activesupport (>= 4.0.0)
+    nio4r (2.7.5)
+    nokogiri (1.19.4-aarch64-linux-gnu)
       racc (~> 1.4)
-    puma (5.6.2)
+    nokogiri (1.19.4-arm64-darwin)
+      racc (~> 1.4)
+    nokogiri (1.19.4-x86_64-darwin)
+      racc (~> 1.4)
+    nokogiri (1.19.4-x86_64-linux-gnu)
+      racc (~> 1.4)
+    omniauth (2.1.4)
+      hashie (>= 3.4.6)
+      logger
+      rack (>= 2.2.3)
+      rack-protection
+    omniauth-rails_csrf_protection (2.0.1)
+      actionpack (>= 4.2)
+      omniauth (~> 2.0)
+    omniauth_openid_connect (0.8.0)
+      omniauth (>= 1.9, < 3)
+      openid_connect (~> 2.2)
+    openid_connect (2.5.0)
+      activemodel
+      attr_required (>= 1.0.0)
+      faraday (~> 2.0)
+      faraday-follow_redirects
+      json-jwt (>= 1.16)
+      mail
+      rack-oauth2 (~> 2.2)
+      swd (~> 2.0)
+      tzinfo
+      validate_url
+      webfinger (~> 2.0)
+    ostruct (0.6.3)
+    parallel (2.2.0)
+    parser (3.3.12.0)
+      ast (~> 2.4.1)
+      racc
+    pp (0.6.4)
+      prettyprint
+    prettyprint (0.2.0)
+    prism (1.9.0)
+    prometheus-client (5.0.0)
+      base64
+    public_suffix (7.0.5)
+    puma (8.0.2)
       nio4r (~> 2.0)
-    racc (1.6.0)
-    rack (2.2.3)
-    rack-test (1.1.0)
-      rack (>= 1.0, < 3)
-    rails (5.2.6.2)
-      actioncable (= 5.2.6.2)
-      actionmailer (= 5.2.6.2)
-      actionpack (= 5.2.6.2)
-      actionview (= 5.2.6.2)
-      activejob (= 5.2.6.2)
-      activemodel (= 5.2.6.2)
-      activerecord (= 5.2.6.2)
-      activestorage (= 5.2.6.2)
-      activesupport (= 5.2.6.2)
-      bundler (>= 1.3.0)
-      railties (= 5.2.6.2)
-      sprockets-rails (>= 2.0.0)
-    rails-dom-testing (2.0.3)
-      activesupport (>= 4.2.0)
+    racc (1.8.1)
+    rack (3.2.7)
+    rack-oauth2 (2.3.0)
+      activesupport
+      attr_required
+      faraday (~> 2.0)
+      faraday-follow_redirects
+      json-jwt (>= 1.11.0)
+      rack (>= 2.1.0)
+    rack-protection (4.2.1)
+      base64 (>= 0.1.0)
+      logger (>= 1.6.0)
+      rack (>= 3.0.0, < 4)
+    rack-session (2.1.2)
+      base64 (>= 0.1.0)
+      rack (>= 3.0.0)
+    rack-test (2.2.0)
+      rack (>= 1.3)
+    rackup (2.3.1)
+      rack (>= 3)
+    rails (7.2.3.2)
+      actioncable (= 7.2.3.2)
+      actionmailbox (= 7.2.3.2)
+      actionmailer (= 7.2.3.2)
+      actionpack (= 7.2.3.2)
+      actiontext (= 7.2.3.2)
+      actionview (= 7.2.3.2)
+      activejob (= 7.2.3.2)
+      activemodel (= 7.2.3.2)
+      activerecord (= 7.2.3.2)
+      activestorage (= 7.2.3.2)
+      activesupport (= 7.2.3.2)
+      bundler (>= 1.15.0)
+      railties (= 7.2.3.2)
+    rails-dom-testing (2.3.0)
+      activesupport (>= 5.0.0)
+      minitest
       nokogiri (>= 1.6)
-    rails-html-sanitizer (1.4.2)
-      loofah (~> 2.3)
-    railties (5.2.6.2)
-      actionpack (= 5.2.6.2)
-      activesupport (= 5.2.6.2)
-      method_source
-      rake (>= 0.8.7)
-      thor (>= 0.19.0, < 2.0)
-    rake (12.3.3)
-    rb-fsevent (0.10.3)
-    rb-inotify (0.9.10)
-      ffi (>= 0.5.0, < 2)
-    resolv (0.2.1)
-    rspec (3.8.0)
-      rspec-core (~> 3.8.0)
-      rspec-expectations (~> 3.8.0)
-      rspec-mocks (~> 3.8.0)
-    rspec-core (3.8.0)
-      rspec-support (~> 3.8.0)
-    rspec-expectations (3.8.1)
+    rails-html-sanitizer (1.7.1)
+      loofah (~> 2.25, >= 2.25.2)
+      nokogiri (>= 1.15.7, != 1.16.7, != 1.16.6, != 1.16.5, != 1.16.4, != 1.16.3, != 1.16.2, != 1.16.1, != 1.16.0.rc1, != 1.16.0)
+    railties (7.2.3.2)
+      actionpack (= 7.2.3.2)
+      activesupport (= 7.2.3.2)
+      cgi
+      irb (~> 1.13)
+      rackup (>= 1.0.0)
+      rake (>= 12.2)
+      thor (~> 1.0, >= 1.2.2)
+      tsort (>= 0.2)
+      zeitwerk (~> 2.6)
+    rainbow (3.1.1)
+    rake (13.4.2)
+    rbs (4.2.0)
+      logger
+      prism (>= 1.6.0)
+      tsort
+    rdoc (8.0.0)
+      erb
+      prism (>= 1.6.0)
+      rbs (>= 4.0.0)
+      tsort
+    regexp_parser (2.12.0)
+    reline (0.7.0)
+      io-console (~> 0.5)
+    resolv (0.7.2)
+    rexml (3.4.4)
+    rouge (4.7.0)
+    rspec (3.13.2)
+      rspec-core (~> 3.13.0)
+      rspec-expectations (~> 3.13.0)
+      rspec-mocks (~> 3.13.0)
+    rspec-core (3.13.6)
+      rspec-support (~> 3.13.0)
+    rspec-expectations (3.13.5)
       diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.8.0)
-    rspec-mocks (3.8.0)
+      rspec-support (~> 3.13.0)
+    rspec-mocks (3.13.8)
       diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.8.0)
-    rspec-rails (3.8.0)
-      actionpack (>= 3.0)
-      activesupport (>= 3.0)
-      railties (>= 3.0)
-      rspec-core (~> 3.8.0)
-      rspec-expectations (~> 3.8.0)
-      rspec-mocks (~> 3.8.0)
-      rspec-support (~> 3.8.0)
-    rspec-support (3.8.0)
-    sass (3.5.7)
-      sass-listen (~> 4.0.0)
-    sass-listen (4.0.0)
-      rb-fsevent (~> 0.9, >= 0.9.4)
-      rb-inotify (~> 0.9, >= 0.9.7)
-    sass-rails (5.0.7)
-      railties (>= 4.0.0, < 6)
-      sass (~> 3.1)
-      sprockets (>= 2.8, < 4.0)
-      sprockets-rails (>= 2.0, < 4.0)
-      tilt (>= 1.1, < 3)
-    secure_headers (6.3.0)
-    secure_random_string (1.0.0)
-    sentry-raven (2.7.4)
-      faraday (>= 0.7.6, < 1.0)
-    sprockets (3.7.2)
-      concurrent-ruby (~> 1.0)
-      rack (> 1, < 3)
-    sprockets-rails (3.4.2)
-      actionpack (>= 5.2)
-      activesupport (>= 5.2)
+      rspec-support (~> 3.13.0)
+    rspec-rails (8.0.4)
+      actionpack (>= 7.2)
+      activesupport (>= 7.2)
+      railties (>= 7.2)
+      rspec-core (>= 3.13.0, < 5.0.0)
+      rspec-expectations (>= 3.13.0, < 5.0.0)
+      rspec-mocks (>= 3.13.0, < 5.0.0)
+      rspec-support (>= 3.13.0, < 5.0.0)
+    rspec-support (3.13.7)
+    rubocop (1.91.0)
+      json (>= 2.3)
+      language_server-protocol (~> 3.17.0.2)
+      lint_roller (~> 1.1.0)
+      parallel (>= 1.10)
+      parser (>= 3.3.0.2)
+      rainbow (>= 2.2.2, < 4.0)
+      regexp_parser (>= 2.9.3, < 3.0)
+      rubocop-ast (>= 1.49.0, < 2.0)
+      ruby-progressbar (~> 1.7)
+      unicode-display_width (>= 2.4.0, < 4.0)
+    rubocop-ast (1.50.0)
+      parser (>= 3.3.7.2)
+      prism (~> 1.7)
+    rubocop-rails (2.37.0)
+      activesupport (>= 4.2.0)
+      lint_roller (~> 1.1)
+      rack (>= 1.1)
+      rubocop (>= 1.89.0, < 2.0)
+      rubocop-ast (>= 1.44.0, < 2.0)
+    ruby-progressbar (1.13.0)
+    sass-rails (6.0.0)
+      sassc-rails (~> 2.1, >= 2.1.1)
+    sassc (2.4.0)
+      ffi (~> 1.9)
+    sassc-rails (2.1.2)
+      railties (>= 4.0.0)
+      sassc (>= 2.0)
+      sprockets (> 3.0)
+      sprockets-rails
+      tilt
+    secure_headers (7.3.0)
+      cgi (>= 0.1)
+    securerandom (0.4.1)
+    sentry-rails (7.0.0)
+      railties (>= 5.2.0)
+      sentry-ruby (~> 7.0.0)
+    sentry-ruby (7.0.0)
+      bigdecimal
+      concurrent-ruby (~> 1.0, >= 1.0.2)
+      logger
+    shoulda-matchers (8.0.1)
+      activesupport (>= 7.2)
+    sprockets (4.4.1)
+      concurrent-ruby (~> 1.1)
+      logger
+      rack (>= 2.2.4, < 4)
+    sprockets-rails (3.5.2)
+      actionpack (>= 6.1)
+      activesupport (>= 6.1)
       sprockets (>= 3.0.0)
-    temple (0.8.2)
-    thor (0.19.4)
-    thread_safe (0.3.6)
-    tilt (2.0.10)
-    turbolinks (5.2.0)
+    swd (2.0.3)
+      activesupport (>= 3)
+      attr_required (>= 0.0.5)
+      faraday (~> 2.0)
+      faraday-follow_redirects
+    temple (0.10.7)
+    thor (1.5.0)
+    tilt (2.9.0)
+    timecop (0.9.11)
+    timeout (0.6.1)
+    tsort (0.2.0)
+    turbolinks (5.2.1)
       turbolinks-source (~> 5.2)
     turbolinks-source (5.2.0)
-    tzinfo (1.2.9)
-      thread_safe (~> 0.1)
-    uglifier (4.1.18)
+    tzinfo (2.0.6)
+      concurrent-ruby (~> 1.0)
+    uglifier (4.2.1)
       execjs (>= 0.3.0, < 3)
-    websocket-driver (0.7.5)
+    unicode-display_width (3.2.0)
+      unicode-emoji (~> 4.1)
+    unicode-emoji (4.2.0)
+    uri (1.1.1)
+    useragent (0.16.11)
+    validate_url (1.0.15)
+      activemodel (>= 3.0.0)
+      public_suffix
+    webfinger (2.1.3)
+      activesupport
+      faraday (~> 2.0)
+      faraday-follow_redirects
+    webmock (3.26.4)
+      addressable (>= 2.8.0)
+      crack (>= 0.3.2)
+      hashdiff (>= 0.4.0, < 2.0.0)
+    webrick (1.9.2)
+    websocket-driver (0.8.2)
+      base64
       websocket-extensions (>= 0.1.0)
     websocket-extensions (0.1.5)
+    zeitwerk (2.8.3)
 
 PLATFORMS
-  ruby
+  aarch64-linux
+  arm64-darwin
+  x86_64-darwin
+  x86_64-linux
 
 DEPENDENCIES
+  abbrev
   annotate
-  authie (~> 3.0)
+  authie
   autoprefixer-rails
-  basic_ssl
   bcrypt
-  bunny
-  byebug
-  changey
   chronic
-  clockwork
-  coffee-rails (~> 4.2)
-  database_cleaner
-  dotenv-rails
-  dynamic_form
-  encrypto_signo
-  factory_bot_rails (~> 4.0)
-  foreman
+  coffee-rails (~> 5.0)
+  database_cleaner-active_record
+  domain_name
+  dotenv
+  execjs
+  factory_bot_rails
   gelf
   haml
   hashie
   highline
   jquery-rails
+  json (< 3)
   jwt
   kaminari
-  mail!
-  moonrope
+  klogger-logger
+  konfig-config (~> 3.0)
+  logger
+  mail
+  mutex_m
   mysql2
   nifty-utils
   nilify_blanks
   nio4r
+  omniauth-rails_csrf_protection
+  omniauth_openid_connect
+  ostruct
+  prometheus-client
   puma
-  rails (= 5.2.6.2)
-  resolv (~> 0.2.1)
+  rackup
+  rails (= 7.2.3.2)
+  resolv
   rspec
   rspec-rails
-  sass-rails (~> 5.0)
+  rubocop
+  rubocop-rails
+  sass-rails
   secure_headers
-  sentry-raven
+  sentry-rails
+  shoulda-matchers
+  timecop
   turbolinks (~> 5)
   uglifier (>= 1.3.0)
+  webmock
+  webrick
 
 BUNDLED WITH
-   1.17.2
+   2.7.2

MIT-LICENCE

@@ -1,4 +1,6 @@
-Copyright 2017 aTech Media Limited.
+Copyright 2017-2024 Krystal Hosting Ltd
+Copyright 2024 Adam Cooke
+Copyright 2024 Charlie Smurthwaite
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

Makefile

@@ -1,115 +0,0 @@
-# ==============================================================================
-# Welcome to the Makefile
-# ==============================================================================
-#
-# This Makefile contains a series of tasks which can help with building, testing
-# and working with the app. The following tasks are available to you:
-#
-# Image building & releasing actions:
-#
-#   make docker-build      - Builds a production Docker image.
-#   make docker-ci-build   - Builds an image without any asset compilation,
-#                            ideal for running test suites and similar tasks.
-#   make docker-image      - Builds a production Docker image and tags it as
-#                            appropriate based on the current branch & tag.
-#   make docker-release    - Builds a production Docker image and uploads to the
-#                            registry.
-#
-
-# ==============================================================================
-# Configuration
-# ==============================================================================
-
-# Docker image name to release the production image as.
-DOCKER_IMAGE := ghcr.io/postalserver/postal
-
-# Path to bundle config
-BUNDLE_CONFIG ?= $(HOME)/.bundle/config
-
-# Detect if a tty is available
-TTY := $(shell [ -t 0 ] && echo 1)
-
-# Tag names
-DOCKER_TAG_NAME ?= $(shell git rev-parse --abbrev-ref HEAD 2>/dev/null)
-DOCKER_TAG_VERSION ?= $(shell git describe --tags --exact-match 2>/dev/null)
-
-ifeq ($(DOCKER_TAG_NAME),master)
-	DOCKER_TAG_NAME = latest
-endif
-
-ifeq ($(DOCKER_TAG_NAME),main)
-	DOCKER_TAG_NAME = latest
-endif
-
-# Version string to use
-VERSION ?= $(DOCKER_TAG_VERSION)
-ifeq ($(VERSION),)
-	VERSION = $(shell git describe --tags 2>/dev/null)
-endif
-
-ifeq ($(VERSION),)
-	VERSION = 0.0.0-dev
-endif
-
-# ==============================================================================
-# Image Building
-# ==============================================================================
-
-DOCKER_BUILD_CMD = DOCKER_BUILDKIT=1 docker \
-	build $(if $(TTY),,--progress plain) \
-	--build-arg VERSION=$(VERSION) \
-	.
-
-DOCKER_CI_BUILD_CMD = $(DOCKER_BUILD_CMD) --target=ci
-
-.PHONY: docker-build
-docker-build:
-	$(DOCKER_BUILD_CMD)
-
-.PHONY: docker-ci-build
-docker-ci-build:
-	$(DOCKER_CI_BUILD_CMD)
-
-# ==============================================================================
-# Image Tagging
-# ==============================================================================
-
-.PHONY: docker-image
-docker-image: docker-build
-	$(eval IMAGE := $(shell $(DOCKER_BUILD_CMD) -q))
-ifeq ($(DOCKER_TAG_NAME),latest)
-	docker tag "$(IMAGE)" "$(DOCKER_IMAGE):$(DOCKER_TAG_NAME)"
-endif
-
-# ==============================================================================
-# Image Releasing
-# ==============================================================================
-
-.PHONY: docker-release
-docker-release: docker-image
-ifeq ($(DOCKER_TAG_NAME),latest)
-	docker push "$(DOCKER_IMAGE):$(DOCKER_TAG_NAME)"
-endif
-ifneq ($(DOCKER_TAG_VERSION),)
-	docker tag "$(DOCKER_IMAGE):$(DOCKER_TAG_NAME)" \
-		"$(DOCKER_IMAGE):$(DOCKER_TAG_VERSION)" && \
-		docker push "$(DOCKER_IMAGE):$(DOCKER_TAG_VERSION)"
-endif
-
-# ==============================================================================
-# Tests
-# ==============================================================================
-
-.PHONY: ci-test
-ci-test: docker-ci-build
-	$(eval IMAGE := $(shell $(DOCKER_CI_BUILD_CMD) -q))
-	$(eval RAND := $(shell echo "$${RANDOM}$$(date +%s)"))
-	POSTAL_IMAGE=$(IMAGE) \
-	docker-compose -p "postal$(RAND)" run --rm postal sh -c 'bundle exec rspec'; \
-	EXIT_CODE=$$?; \
-	docker-compose -p "postal$(RAND)" down -v; \
-	exit $$EXIT_CODE
-
-.PHONY: test
-test:
-	bundle exec rspec

Procfile.dev

@@ -1,5 +1,3 @@
-web: bundle exec puma -C config/puma.rb
+web: unset PORT; bundle exec puma -C config/puma.rb
 worker: bundle exec ruby script/worker.rb
-cron: bundle exec rake postal:cron
-smtp: bundle exec rake postal:smtp_server
-requeuer: bundle exec rake postal:requeuer
+smtp: unset PORT; bundle exec ruby script/smtp_server.rb

README.md

@@ -1,10 +1,9 @@
-<img src="https://share.adam.ac/21/logo-g3hWemROTemCYbWlmL.svg" width="250" />
+![GitHub Header](https://github.com/postalserver/.github/assets/4765/7a63c35d-2f47-412f-a6b3-aebc92a55310)
 
-**Postal** is a complete and fully featured mail server for use by websites & web servers. Think Sendgrid, Mailgun or Postmark but open source and ready for you to run on your own servers. Postal is developed by [Krystal](https://k.io) to serve its own mail processing requirements and we have since decided that it should be released as an open source project for the community.
-
-![Screenshot](https://share.adam.ac/17/k4lA5OuPlU2.png)
+**Postal** is a complete and fully featured mail server for use by websites & web servers. Think Sendgrid, Mailgun or Postmark but open source and ready for you to run on your own servers. 
 
 * [Documentation](https://docs.postalserver.io)
-* [Installation Instructions](https://docs.postalserver.io/install/prerequisites)
+* [Installation Instructions](https://docs.postalserver.io/getting-started)
 * [FAQs](https://docs.postalserver.io/welcome/faqs) & [Features](https://docs.postalserver.io/welcome/feature-list)
-* [Ask for help](https://github.com/postalserver/postal/discussions)
+* [Discussions](https://github.com/postalserver/postal/discussions) - ask for help or request a feature
+* [Join us on Discord](https://discord.postalserver.io)

Rakefile

@@ -1,6 +1,8 @@
+# frozen_string_literal: true
+
 # Add your own tasks in files placed in lib/tasks ending in .rake,
 # for example lib/tasks/capistrano.rake, and they will automatically be available to Rake.
 
-require_relative 'config/application'
+require_relative "config/application"
 
 Rails.application.load_tasks

SECURITY.md

@@ -2,14 +2,14 @@
 
 ## Supported Versions
 
-We only support updates to the 2.x versions of Postal.
+We only support updates to the 3.x versions of Postal.
 
 | Version | Supported          |
 | ------- | ------------------ |
-| 2.x.x   | :white_check_mark: |
-| < 2.0   | :x:                |
+| 3.x.x   | :white_check_mark: |
+| < 3.0   | :x:                |
 
 ## Reporting a Vulnerability
 
-If you discover a vulnerability in Postal, please do not post an issue on GitHub. Instead you should send an
-e-mail to security@postalserver.io with details. We will get back to you directly.
+If you discover a vulnerability in Postal, please do not post an issue on GitHub. Please,
+instead, [create a new security advisory through GitHub](https://github.com/postalserver/postal/security/advisories/new).

api/authenticator.rb

@@ -1,29 +0,0 @@
-authenticator :server do
-  friendly_name "Server Authenticator"
-  header "X-Server-API-Key", "The API token for a server that you wish to authenticate with.", :example => 'f29a45f0d4e1744ebaee'
-  error 'InvalidServerAPIKey', "The API token provided in X-Server-API-Key was not valid.", :attributes => {:token => "The token that was looked up"}
-  error 'ServerSuspended', "The mail server has been suspended"
-  lookup do
-    if key = request.headers['X-Server-API-Key']
-      if credential = Credential.where(:type => 'API', :key => key).first
-        if credential.server.suspended?
-          error 'ServerSuspended'
-        else
-          credential.use
-          credential
-        end
-      else
-        error 'InvalidServerAPIKey', :token => key
-      end
-    end
-  end
-  rule :default, "AccessDenied", "Must be authenticated as a server." do
-    identity.is_a?(Credential)
-  end
-end
-
-authenticator :anonymous do
-  rule :default, "MustNotBeAuthenticated", "Must not be authenticated." do
-    identity.nil?
-  end
-end

api/controllers/messages_api_controller.rb

@@ -1,40 +0,0 @@
-controller :messages do
-  friendly_name "Messages API"
-  description "This API allows you to access message details"
-  authenticator :server
-
-  action :message do
-    title "Return message details"
-    description "Returns all details about a message"
-    param :id, "The ID of the message", :type => Integer, :required => true
-    returns Hash, :structure => :message, :structure_opts => {:paramable => {:expansions => false}}
-    error 'MessageNotFound', "No message found matching provided ID", :attributes => {:id => "The ID of the message"}
-    action do
-      begin
-        message = identity.server.message(params.id)
-      rescue Postal::MessageDB::Message::NotFound => e
-        error 'MessageNotFound', :id => params.id
-      end
-      structure :message, message, :return => true
-    end
-  end
-
-  action :deliveries do
-    title "Return deliveries for a message"
-    description "Returns an array of deliveries which have been attempted for this message"
-    param :id, "The ID of the message", :type => Integer, :required => true
-    returns Array, :structure => :delivery, :structure_opts => {:full => true}
-    error 'MessageNotFound', "No message found matching provided ID", :attributes => {:id => "The ID of the message"}
-    action do
-      begin
-        message = identity.server.message(params.id)
-      rescue Postal::MessageDB::Message::NotFound => e
-        error 'MessageNotFound', :id => params.id
-      end
-      message.deliveries.map do |d|
-        structure :delivery, d
-      end
-    end
-  end
-
-end

api/controllers/send_api_controller.rb

@@ -1,110 +0,0 @@
-controller :send do
-  friendly_name "Send API"
-  description "This API allows you to send messages"
-  authenticator :server
-
-  action :message do
-    title "Send a message"
-    description "This action allows you to send a message by providing the appropriate options"
-    # Acceptable Parameters
-    param :to, "The e-mail addresses of the recipients (max 50)", :type => Array
-    param :cc, "The e-mail addresses of any CC contacts (max 50)", :type => Array
-    param :bcc, "The e-mail addresses of any BCC contacts (max 50)", :type => Array
-    param :from, "The e-mail address for the From header", :type => String
-    param :sender, "The e-mail address for the Sender header", :type => String
-    param :subject, "The subject of the e-mail", :type => String
-    param :tag, "The tag of the e-mail", :type => String
-    param :reply_to, "Set the reply-to address for the mail", :type => String
-    param :plain_body, "The plain text body of the e-mail", :type => String
-    param :html_body, "The HTML body of the e-mail", :type => String
-    param :attachments, "An array of attachments for this e-mail", :type => Array
-    param :headers, "A hash of additional headers", :type => Hash
-    param :bounce, "Is this message a bounce?", :type => :boolean
-    # Errors
-    error 'ValidationError', "The provided data was not sufficient to send an email", :attributes => {:errors => "A hash of error details"}
-    error 'NoRecipients', "There are no recipients defined to receive this message"
-    error 'NoContent', "There is no content defined for this e-mail"
-    error 'TooManyToAddresses', "The maximum number of To addresses has been reached (maximum 50)"
-    error 'TooManyCCAddresses', "The maximum number of CC addresses has been reached (maximum 50)"
-    error 'TooManyBCCAddresses', "The maximum number of BCC addresses has been reached (maximum 50)"
-    error 'FromAddressMissing', "The From address is missing and is required"
-    error 'UnauthenticatedFromAddress', "The From address is not authorised to send mail from this server"
-    error 'AttachmentMissingName', "An attachment is missing a name"
-    error 'AttachmentMissingData', "An attachment is missing data"
-    # Return
-    returns Hash
-    # Action
-    action do
-      attributes = {}
-      attributes[:to] = params.to
-      attributes[:cc] = params.cc
-      attributes[:bcc] = params.bcc
-      attributes[:from] = params.from
-      attributes[:sender] = params.sender
-      attributes[:subject] = params.subject
-      attributes[:reply_to] = params.reply_to
-      attributes[:plain_body] = params.plain_body
-      attributes[:html_body] = params.html_body
-      attributes[:bounce] = params.bounce ? true : false
-      attributes[:tag] = params.tag
-      attributes[:custom_headers] = params.headers
-      attributes[:attachments] = []
-      (params.attachments || []).each do |attachment|
-        next unless attachment.is_a?(Hash)
-        attributes[:attachments] << {:name => attachment['name'], :content_type => attachment['content_type'], :data => attachment['data'], :base64 => true}
-      end
-      message = OutgoingMessagePrototype.new(identity.server, request.ip, 'api', attributes)
-      message.credential = identity
-      if message.valid?
-        result = message.create_messages
-        {:message_id => message.message_id, :messages => result}
-      else
-        error message.errors.first
-      end
-    end
-  end
-
-  action :raw do
-    title "Send a raw RFC2882 message"
-    description "This action allows you to send us a raw RFC2822 formatted message along with the recipients that it should be sent to. This is similar to sending a message through our SMTP service."
-    param :mail_from, "The address that should be logged as sending the message", :type => String, :required => true
-    param :rcpt_to, "The addresses this message should be sent to", :type => Array, :required => true
-    param :data, "A base64 encoded RFC2822 message to send", :type => String, :required => true
-    param :bounce, "Is this message a bounce?", :type => :boolean
-    returns Hash
-    error 'UnauthenticatedFromAddress', "The From address is not authorised to send mail from this server"
-    action do
-      # Decode the raw message
-      raw_message = Base64.decode64(params.data)
-
-      # Parse through mail to get the from/sender headers
-      mail = Mail.new(raw_message.split("\r\n\r\n", 2).first)
-      from_headers = {'from' => mail.from, 'sender' => mail.sender}
-      authenticated_domain = identity.server.find_authenticated_domain_from_headers(from_headers)
-
-      # If we're not authenticated, don't continue
-      if authenticated_domain.nil?
-        error 'UnauthenticatedFromAddress'
-      end
-
-      # Store the result ready to return
-      result = {:message_id => nil, :messages => {}}
-      params.rcpt_to.uniq.each do |rcpt_to|
-        message = identity.server.message_db.new_message
-        message.rcpt_to = rcpt_to
-        message.mail_from = params.mail_from
-        message.raw_message = raw_message
-        message.received_with_ssl = true
-        message.scope = 'outgoing'
-        message.domain_id = authenticated_domain.id
-        message.credential_id = identity.id
-        message.bounce = params.bounce ? 1 : 0
-        message.save
-        result[:message_id] = message.message_id if result[:message_id].nil?
-        result[:messages][rcpt_to] = {:id => message.id, :token => message.token}
-      end
-      result
-    end
-  end
-
-end

api/structures/delivery_api_structure.rb

@@ -1,10 +0,0 @@
-structure :delivery do
-  basic :id
-  basic :status
-  basic :details
-  basic :output, :value => proc { o.output&.strip }
-  basic :sent_with_ssl, :value => proc { o.sent_with_ssl == 1 }
-  basic :log_id
-  basic :time, :value => proc { o.time&.to_f }
-  basic :timestamp, :value => proc { o.timestamp.to_f }
-end

api/structures/message_api_structure.rb

@@ -1,66 +0,0 @@
-structure :message do
-  basic :id
-  basic :token
-
-  expansion(:status) {
-    {
-      :status => o.status,
-      :last_delivery_attempt => o.last_delivery_attempt ? o.last_delivery_attempt.to_f : nil,
-      :held => o.held == 1 ? true : false,
-      :hold_expiry => o.hold_expiry ? o.hold_expiry.to_f : nil
-    }
-  }
-
-  expansion(:details) {
-    {
-      :rcpt_to => o.rcpt_to,
-      :mail_from => o.mail_from,
-      :subject => o.subject,
-      :message_id => o.message_id,
-      :timestamp => o.timestamp.to_f,
-      :direction => o.scope,
-      :size => o.size,
-      :bounce => o.bounce,
-      :bounce_for_id => o.bounce_for_id,
-      :tag => o.tag,
-      :received_with_ssl => o.received_with_ssl
-    }
-  }
-
-  expansion(:inspection) {
-    {
-      :inspected => o.inspected == 1 ? true : false,
-      :spam => o.spam == 1 ? true : false,
-      :spam_score => o.spam_score.to_f,
-      :threat => o.threat == 1 ? true : false,
-      :threat_details => o.threat_details
-    }
-  }
-
-  expansion(:plain_body) { o.plain_body }
-
-  expansion(:html_body) { o.html_body }
-
-  expansion(:attachments) {
-    o.attachments.map do |attachment|
-      {
-        :filename => attachment.filename.to_s,
-        :content_type => attachment.mime_type,
-        :data => Base64.encode64(attachment.body.to_s),
-        :size => attachment.body.to_s.bytesize,
-        :hash => Digest::SHA1.hexdigest(attachment.body.to_s)
-      }
-    end
-  }
-
-  expansion(:headers) { o.headers }
-
-  expansion(:raw_message) { Base64.encode64(o.raw_message) }
-  
-  expansion(:activity_entries) {
-    {
-      :loads => o.loads,
-      :clicks => o.clicks
-    }
-  }
-end

app/assets/config/manifest.js

@@ -1,3 +1,5 @@
 //= link_tree ../images
 //= link_directory ../javascripts .js
 //= link_directory ../stylesheets .css
+//= link application/application.css
+//= link application/application.js

app/assets/stylesheets/application/components/_error_explanation.scss

@@ -9,14 +9,6 @@
   line-height:1.5;
 }
 
-.errorExplanation h2 {
-  display:none;
-}
-
-.errorExplanation p {
-  display:none;
-}
-
 .errorExplanation ul li {
   list-style:disc;
   margin-left:20px;

app/assets/stylesheets/application/components/_login_form.scss

@@ -1,22 +1,32 @@
-.loginForm {
-
-}
+.loginForm {}
 
 
 .loginForm__input {
-  margin-bottom:15px;
+  margin-bottom: 15px;
 }
 
 .loginForm__submit {
-  display:flex;
-  justify-content:space-between;
-  align-items:center;
+  display: flex;
+  justify-content: space-between;
+  align-items: center;
 }
 
 
 .loginForm__links {
-  font-size:12px;
-  color:#999;
+  font-size: 12px;
+  color: #999;
   text-decoration: underline;
-  line-height:1.7;
+  line-height: 1.7;
+}
+
+.loginForm__divider {
+  margin-top: 25px;
+  margin-bottom: 25px;
+  border-top: 1px solid #e4e8ef;
+}
+
+.loginForm__localTitle {
+  text-align: center;
+  margin-bottom: 15px;
+  color: #999;
 }

app/assets/stylesheets/application/components/_site_content.scss

@@ -6,7 +6,6 @@
 }
 
 .siteContent__main {
-  @include scrollbars(12px);
   flex: 1 1 auto;
   z-index:100;
   overflow-y:scroll;

app/assets/stylesheets/application/components/_site_header.scss

@@ -4,11 +4,6 @@
   flex: 0 0 auto;
   color:#fff;
   z-index:1000;
-  border-right:12px solid #efefef;
-
-  .browser-firefox & {
-    border-right: 15px solid #fafafa;
-  }
 }
 
 .siteHeader__inside {

app/assets/stylesheets/application/components/_user_list.scss

@@ -1,74 +1,67 @@
 .userList {
-  border-radius:4px;
-  color:$darkBlue;
-  overflow:hidden;
-  box-shadow:0 0 10px rgba(0,0,0,0.2);
+  border-radius: 4px;
+  color: $darkBlue;
+  overflow: hidden;
+  box-shadow: 0 0 10px rgba(0, 0, 0, 0.2);
 }
 
 .userList__item {
-  display:block;
-  background:#fff;
-  padding:15px;
-  display:flex;
+  display: block;
+  background: #fff;
+  padding: 15px;
+  display: flex;
   align-items: center;
 }
+
 .userList__item:nth-child(even) {
-  background:none;
+  background: none;
 }
 
-.userList__item + .userList__item {
-  border-top:1px solid lighten(#ccd4e0, 10%);
-}
-
-.userList__avatar {
-  width:50px;
-  height:50px;
-  border-radius:50%;
-  background:#fff;
-  border:2px solid #efefef;
-  padding:3px;
-  flex: 0 0 auto;
+.userList__item+.userList__item {
+  border-top: 1px solid lighten(#ccd4e0, 10%);
 }
 
 .userList__details {
   flex: 1 1 auto;
-  margin:0 25px;
+  margin: 0 0;
 }
 
 
 .userList__actions {
   flex: 0 0 auto;
-  width:180px;
-  font-size:12px;
-  line-height:1.5;
-  color:#999;
-  text-decoration: underline;
+  width: 120px;
+  font-size: 12px;
+  line-height: 1.5;
+  color: #999;
+
+  a {
+    text-decoration: underline;
+  }
 }
 
 .userList__name {
-  font-weight:600;
-  font-size:16px;
-  margin-bottom:3px;
+  font-weight: 600;
+  font-size: 16px;
+  margin-bottom: 3px;
 }
 
 .userList__owner {
-  vertical-align:2px;
-  margin-left:5px;
-  background-color:$orange;
+  vertical-align: 2px;
+  margin-left: 5px;
+  background-color: $orange;
 }
 
 .userList__pending {
-  vertical-align:2px;
-  margin-left:5px;
-  background-color:#ccc;
+  vertical-align: 2px;
+  margin-left: 5px;
+  background-color: #ccc;
 }
 
-.userList__admin {
-  vertical-align:2px;
-  margin-left:5px;
-  background-color:$blue;
+.userList__tag {
+  vertical-align: 2px;
+  margin-left: 3px;
 }
 
 .userList__revoke {
-  color:$red;
+  color: $red;
 }

app/assets/stylesheets/application/elements/_button.scss

@@ -1,98 +1,117 @@
 .button {
-  display:inline-block;
-  font:inherit;
-  border-radius:4px;
-  appearance:none;
-  background:$blue;
-  color:#fff;
-  font-size:14px !important;
-  margin:0;
-  vertical-align:top;
-  padding:6px 15px;
-  border:2px solid transparent;
-  border-bottom:2px solid darken($blue, 20%);
+  display: inline-block;
+  font: inherit;
+  border-radius: 4px;
+  appearance: none;
+  background: $blue;
+  color: #fff;
+  font-size: 14px !important;
+  margin: 0;
+  vertical-align: top;
+  padding: 6px 15px;
+  border: 2px solid transparent;
+  border-bottom: 2px solid darken($blue, 20%);
+
   &:active {
-    background-color:darken($blue, 15%);
+    background-color: darken($blue, 15%);
   }
+
   &:focus {
-    border-color:darken($blue, 15%);
-    background-color:lighten($blue, 5%);
+    border-color: darken($blue, 15%);
+    background-color: lighten($blue, 5%);
   }
+
   &.is-spinning {
-    color:transparent;
-    background-repeat:no-repeat;
+    color: transparent;
+    background-repeat: no-repeat;
     background-position: center center;
-    background-size:25px;
-    background-image:image-url('button-spinner.gif');
+    background-size: 25px;
+    background-image: image-url('button-spinner.gif');
   }
 }
 
 .button--small {
-  font-size:12px !important;
-  padding:3px 10px;
-  border-width:1px;
+  font-size: 12px !important;
+  padding: 3px 10px;
+  border-width: 1px;
 }
 
 .button--positive {
-  background-color:$green;
-  border-bottom-color:darken($green, 15%);
+  background-color: $green;
+  border-bottom-color: darken($green, 15%);
+
   &:active {
-    background-color:darken($green, 15%);
+    background-color: darken($green, 15%);
   }
+
   &:focus {
-    border-color:darken($green, 15%);
-    background-color:lighten($green, 5%);
+    border-color: darken($green, 15%);
+    background-color: lighten($green, 5%);
   }
+
   &.is-spinning {
-    background-image:image-url('button-spinner-positive.gif');
+    background-image: image-url('button-spinner-positive.gif');
   }
 
 }
 
 
 .button--neutral {
-  background-color:#ccc;
-  border-bottom-color:darken(#ccc, 15%);
+  background-color: #ccc;
+  border-bottom-color: darken(#ccc, 15%);
+
   &:active {
-    background-color:darken(#ccc, 15%);
+    background-color: darken(#ccc, 15%);
   }
+
   &:focus {
-    border-color:darken(#ccc, 15%);
-    background-color:lighten(#ccc, 5%);
+    border-color: darken(#ccc, 15%);
+    background-color: lighten(#ccc, 5%);
   }
+
   &.is-spinning {
-    background-image:image-url('button-spinner-neutral.gif');
+    background-image: image-url('button-spinner-neutral.gif');
   }
 }
 
 .button--danger {
-  background-color:$red;
-  border-bottom-color:darken($red, 15%);
+  background-color: $red;
+  border-bottom-color: darken($red, 15%);
+
   &:active {
-    background-color:darken($red, 15%);
+    background-color: darken($red, 15%);
   }
+
   &:focus {
-    border-color:darken($red, 15%);
-    background-color:lighten($red, 5%);
+    border-color: darken($red, 15%);
+    background-color: lighten($red, 5%);
   }
+
   &.is-spinning {
-    background-image:image-url('button-spinner-danger.gif');
+    background-image: image-url('button-spinner-danger.gif');
   }
 }
 
 .button--dark {
-  background-color:$darkBlue;
-  border-bottom-color:darken($darkBlue, 15%);
+  background-color: $darkBlue;
+  border-bottom-color: darken($darkBlue, 15%);
+
   &:active {
-    background-color:darken($darkBlue, 15%);
+    background-color: darken($darkBlue, 15%);
   }
+
   &:focus {
-    border-color:darken($darkBlue, 15%);
-    background-color:lighten($darkBlue, 5%);
+    border-color: darken($darkBlue, 15%);
+    background-color: lighten($darkBlue, 5%);
   }
+
   &.is-spinning {
-    background-image:image-url('button-spinner-dark.gif');
+    background-image: image-url('button-spinner-dark.gif');
   }
 
 }
 
+.button--full {
+  width: 100%;
+  text-align: center;
+}

app/assets/stylesheets/application/elements/_label.scss

@@ -1,136 +1,137 @@
 .label {
-  display:inline-block;
-  background:#000;
-  color:#fff;
-  font-size:9px;
+  display: inline-block;
+  background: #000;
+  color: #fff;
+  font-size: 9px;
   text-transform: uppercase;
-  border-radius:40px;
-  padding:2px 6px;
-  line-height:0.9;
+  border-radius: 40px;
+  padding: 2px 6px;
+  line-height: 0.9;
 }
 
 .label--green {
-  background-color:$green;
+  background-color: $green;
 }
 
 .label--red {
-  background-color:$red;
+  background-color: $red;
 }
 
 .label--orange {
-  background-color:$orange;
+  background-color: $orange;
 }
 
 .label--blue {
-  background-color:$blue;
+  background-color: $blue;
 }
 
 .label--grey {
-  background-color:#999;
+  background-color: #999;
 }
 
 .label--turquoise {
-  background-color:$blue;
+  background-color: $blue;
 }
 
 .label--purple {
-  background-color:$purple;
+  background-color: $purple;
 }
 
 .label--large {
-  font-size:11px;
-  padding:4px 10px;
+  font-size: 11px;
+  padding: 4px 10px;
 }
 
 .label--serverStatus-live {
-  background-color:$green;
+  background-color: $green;
 }
 
 .label--serverStatus-development {
-  background-color:#636363;
+  background-color: #636363;
 }
 
 .label--serverStatus-suspended {
-  background-color:$red;
+  background-color: $red;
 }
 
 .label--messageStatus-pending {
-  background-color:$subBlue;
+  background-color: $subBlue;
 }
 
 .label--messageStatus-held {
-  background-color:#aaa;
+  background-color: #aaa;
 }
 
 .label--messageStatus-processed {
-  background-color:$green;
+  background-color: $green;
 }
 
 .label--messageStatus-sent {
-  background-color:$green;
+  background-color: $green;
 }
 
 .label--messageStatus-hard_fail {
-  background-color:$red;
+  background-color: $red;
 }
 
 .label--messageStatus-soft_fail {
-  background-color:$orange;
+  background-color: $orange;
 }
 
 .label--messageStatus-bounced {
-  background-color:$red;
+  background-color: $red;
 }
 
 .label--messageStatus-hold_cancelled {
-  background-color:#ccc;
+  background-color: #ccc;
 }
 
 
 .label--credentialType-api {
-  background-color:$blue;
+  background-color: $blue;
 }
 
 .label--credentialType-smtp {
-  background-color:$turquoise;
+  background-color: $turquoise;
 }
 
 .label--credentialType-smtp_ip {
-  background-color:$orange;
+  background-color: $orange;
 }
 
 .label--spamStatus-not_checked {
-  background:#aaa;
+  background: #aaa;
 }
 
 .label--spamStatus-spam {
-  background:$orange;
+  background: $orange;
 }
 
 .label--spamStatus-not_spam {
-  background:$turquoise;
+  background: $turquoise;
 }
 
 .label--http-status-2 {
-  background-color:$green;
+  background-color: $green;
 }
 
 .label--http-status-3 {
-  background-color:$orange;
+  background-color: $orange;
 }
 
 .label--http-status-4,
 .label--http-status-5 {
-  background-color:$red;
+  background-color: $red;
 }
 
 .domainList__ssl {
-  color:$green;
+  color: $green;
+
   &:hover {
-    text-decoration:underline;
+    text-decoration: underline;
   }
 }
 
 .domainList__ssl--disabled {
-  color:#999;
+  color: #999;
 }

app/controllers/address_endpoints_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class AddressEndpointsController < ApplicationController
 
   include WithinOrganization
@@ -19,7 +21,7 @@ class AddressEndpointsController < ApplicationController
       flash[:notice] = params[:return_notice] if params[:return_notice].present?
       redirect_to_with_json [:return_to, [organization, @server, :address_endpoints]]
     else
-      render_form_errors 'new', @address_endpoint
+      render_form_errors "new", @address_endpoint
     end
   end
 
@@ -27,7 +29,7 @@ class AddressEndpointsController < ApplicationController
     if @address_endpoint.update(safe_params)
       redirect_to_with_json [organization, @server, :address_endpoints]
     else
-      render_form_errors 'edit', @address_endpoint
+      render_form_errors "edit", @address_endpoint
     end
   end
 

app/controllers/application_controller.rb

@@ -1,4 +1,6 @@
-require 'authie/session'
+# frozen_string_literal: true
+
+require "authie/session"
 
 class ApplicationController < ActionController::Base
 
@@ -7,37 +9,37 @@ class ApplicationController < ActionController::Base
   before_action :login_required
   before_action :set_timezone
 
-  rescue_from Authie::Session::InactiveSession, :with => :auth_session_error
-  rescue_from Authie::Session::ExpiredSession, :with => :auth_session_error
-  rescue_from Authie::Session::BrowserMismatch, :with => :auth_session_error
+  rescue_from Authie::Session::InactiveSession, with: :auth_session_error
+  rescue_from Authie::Session::ExpiredSession, with: :auth_session_error
+  rescue_from Authie::Session::BrowserMismatch, with: :auth_session_error
 
   private
 
   def login_required
-    unless logged_in?
-      redirect_to login_path(:return_to => request.fullpath)
-    end
+    return if logged_in?
+
+    redirect_to login_path(return_to: request.fullpath)
   end
 
   def admin_required
     if logged_in?
       unless current_user.admin?
-        render :plain => "Not permitted"
+        render plain: "Not permitted"
       end
     else
-      redirect_to login_path(:return_to => request.fullpath)
+      redirect_to login_path(return_to: request.fullpath)
     end
   end
 
   def require_organization_owner
-    unless organization.owner == current_user
-      redirect_to organization_root_path(organization), :alert => "This page can only be accessed by the organization's owner (#{organization.owner.name})"
-    end
+    return if organization.owner == current_user
+
+    redirect_to organization_root_path(organization), alert: "This page can only be accessed by the organization's owner (#{organization.owner.name})"
   end
 
   def auth_session_error(exception)
     Rails.logger.info "AuthSessionError: #{exception.class}: #{exception.message}"
-    redirect_to login_path(:return_to => request.fullpath)
+    redirect_to login_path(return_to: request.fullpath)
   end
 
   def page_title
@@ -46,15 +48,11 @@ class ApplicationController < ActionController::Base
   helper_method :page_title
 
   def redirect_to_with_return_to(url, *args)
-    if params[:return_to].blank? || !params[:return_to].starts_with?('/')
-      redirect_to url_with_return_to(url), *args
-    else
-      redirect_to url_with_return_to(url), *args
-    end
+    redirect_to url_with_return_to(url), *args
   end
 
   def set_timezone
-    Time.zone = logged_in? ? current_user.time_zone : 'UTC'
+    Time.zone = logged_in? ? current_user.time_zone : "UTC"
   end
 
   def append_info_to_payload(payload)
@@ -64,10 +62,13 @@ class ApplicationController < ActionController::Base
   end
 
   def url_with_return_to(url)
-    if params[:return_to].blank? || !params[:return_to].starts_with?('/')
+    return_to = params[:return_to]
+    if return_to.blank? ||
+       !return_to.start_with?("/") ||
+       return_to.start_with?("//", "/\\")
       url_for(url)
     else
-      params[:return_to]
+      return_to
     end
   end
 
@@ -83,14 +84,14 @@ class ApplicationController < ActionController::Base
     end
     respond_to do |wants|
       wants.html { redirect_to url }
-      wants.json { render :json => {:redirect_to => url} }
+      wants.json { render json: { redirect_to: url } }
     end
   end
 
   def render_form_errors(action_name, object)
     respond_to do |wants|
       wants.html { render action_name }
-      wants.json { render :json => {:form_errors => object.errors.full_messages}, :status => 422 }
+      wants.json { render json: { form_errors: object.errors.map(&:full_message) }, status: :unprocessable_entity }
     end
   end
 
@@ -102,7 +103,7 @@ class ApplicationController < ActionController::Base
           render options[:render_action]
         end
       end
-      wants.json { render :json => {:flash => {type => message}} }
+      wants.json { render json: { flash: { type => message } } }
     end
   end
 
@@ -111,7 +112,8 @@ class ApplicationController < ActionController::Base
       auth_session.invalidate!
       reset_session
     end
-    Authie::Session.start(self, :user => user)
+
+    create_auth_session(user)
     @current_user = user
   end
 

app/controllers/concerns/within_organization.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module WithinOrganization
 
   extend ActiveSupport::Concern

app/controllers/credentials_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class CredentialsController < ApplicationController
 
   include WithinOrganization
@@ -18,7 +20,7 @@ class CredentialsController < ApplicationController
     if @credential.save
       redirect_to_with_json [organization, @server, :credentials]
     else
-      render_form_errors 'new', @credential
+      render_form_errors "new", @credential
     end
   end
 
@@ -26,7 +28,7 @@ class CredentialsController < ApplicationController
     if @credential.update(params.require(:credential).permit(:name, :key, :hold))
       redirect_to_with_json [organization, @server, :credentials]
     else
-      render_form_errors 'edit', @credential
+      render_form_errors "edit", @credential
     end
   end
 

app/controllers/domains_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class DomainsController < ApplicationController
 
   include WithinOrganization
@@ -28,7 +30,7 @@ class DomainsController < ApplicationController
     @domain = scope.build(params.require(:domain).permit(:name, :verification_method))
 
     if current_user.admin?
-      @domain.verification_method = 'DNS'
+      @domain.verification_method = "DNS"
       @domain.verified_at = Time.now
     end
 
@@ -39,7 +41,7 @@ class DomainsController < ApplicationController
         redirect_to_with_json [:verify, organization, @server, @domain]
       end
     else
-      render_form_errors 'new', @domain
+      render_form_errors "new", @domain
     end
   end
 
@@ -50,56 +52,57 @@ class DomainsController < ApplicationController
 
   def verify
     if @domain.verified?
-      redirect_to [organization, @server, :domains], :alert => "#{@domain.name} has already been verified."
+      redirect_to [organization, @server, :domains], alert: "#{@domain.name} has already been verified."
       return
     end
 
-    if request.post?
-      case @domain.verification_method
-      when 'DNS'
-        if @domain.verify_with_dns
-          redirect_to_with_json [:setup, organization, @server, @domain], :notice => "#{@domain.name} has been verified successfully. You now need to configure your DNS records."
+    return unless request.post?
+
+    case @domain.verification_method
+    when "DNS"
+      if @domain.verify_with_dns
+        redirect_to_with_json [:setup, organization, @server, @domain], notice: "#{@domain.name} has been verified successfully. You now need to configure your DNS records."
+      else
+        respond_to do |wants|
+          wants.html { flash.now[:alert] = "We couldn't verify your domain. Please double check you've added the TXT record correctly." }
+          wants.json { render json: { flash: { alert: "We couldn't verify your domain. Please double check you've added the TXT record correctly." } } }
+        end
+      end
+    when "Email"
+      if params[:code]
+        if @domain.verification_token == params[:code].to_s.strip
+          @domain.mark_as_verified
+          redirect_to_with_json [:setup, organization, @server, @domain], notice: "#{@domain.name} has been verified successfully. You now need to configure your DNS records."
         else
           respond_to do |wants|
-            wants.html { flash.now[:alert] = "We couldn't verify your domain. Please double check you've added the TXT record correctly." }
-            wants.json { render :json => {:flash => {:alert => "We couldn't verify your domain. Please double check you've added the TXT record correctly."}}}
+            wants.html { flash.now[:alert] = "Invalid verification code. Please check and try again." }
+            wants.json { render json: { flash: { alert: "Invalid verification code. Please check and try again." } } }
           end
         end
-      when 'Email'
-        if params[:code]
-          if @domain.verification_token == params[:code].to_s.strip
-            @domain.verify
-            redirect_to_with_json [:setup, organization, @server, @domain], :notice => "#{@domain.name} has been verified successfully. You now need to configure your DNS records."
-          else
-            respond_to do |wants|
-              wants.html { flash.now[:alert] = "Invalid verification code. Please check and try again." }
-              wants.json { render :json => {:flash => {:alert => "Invalid verification code. Please check and try again."}}}
-            end
-          end
-        elsif params[:email_address].present?
-          raise Postal::Error, "Invalid email address" unless @domain.verification_email_addresses.include?(params[:email_address])
-          AppMailer.verify_domain(@domain, params[:email_address], current_user).deliver
-          if @domain.owner.is_a?(Server)
-            redirect_to_with_json verify_organization_server_domain_path(organization, @server, @domain, :email_address => params[:email_address])
-          else
-            redirect_to_with_json verify_organization_domain_path(organization, @domain, :email_address => params[:email_address])
-          end
+      elsif params[:email_address].present?
+        raise Postal::Error, "Invalid email address" unless @domain.verification_email_addresses.include?(params[:email_address])
+
+        AppMailer.verify_domain(@domain, params[:email_address], current_user).deliver
+        if @domain.owner.is_a?(Server)
+          redirect_to_with_json verify_organization_server_domain_path(organization, @server, @domain, email_address: params[:email_address])
+        else
+          redirect_to_with_json verify_organization_domain_path(organization, @domain, email_address: params[:email_address])
         end
       end
     end
   end
 
   def setup
-    unless @domain.verified?
-      redirect_to [:verify, organization, @server, @domain], :alert => "You can't set up DNS for this domain until it has been verified."
-    end
+    return if @domain.verified?
+
+    redirect_to [:verify, organization, @server, @domain], alert: "You can't set up DNS for this domain until it has been verified."
   end
 
   def check
     if @domain.check_dns(:manual)
-      redirect_to_with_json [organization, @server, :domains], :notice => "Your DNS records for #{@domain.name} look good!"
+      redirect_to_with_json [organization, @server, :domains], notice: "Your DNS records for #{@domain.name} look good!"
     else
-      redirect_to_with_json [:setup, organization, @server, @domain], :alert => "There seems to be something wrong with your DNS records. Check below for information."
+      redirect_to_with_json [:setup, organization, @server, @domain], alert: "There seems to be something wrong with your DNS records. Check below for information."
     end
   end
 

app/controllers/help_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class HelpController < ApplicationController
 
   include WithinOrganization

app/controllers/http_endpoints_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class HTTPEndpointsController < ApplicationController
 
   include WithinOrganization
@@ -19,7 +21,7 @@ class HTTPEndpointsController < ApplicationController
       flash[:notice] = params[:return_notice] if params[:return_notice].present?
       redirect_to_with_json [:return_to, [organization, @server, :http_endpoints]]
     else
-      render_form_errors 'new', @http_endpoint
+      render_form_errors "new", @http_endpoint
     end
   end
 
@@ -27,7 +29,7 @@ class HTTPEndpointsController < ApplicationController
     if @http_endpoint.update(safe_params)
       redirect_to_with_json [organization, @server, :http_endpoints]
     else
-      render_form_errors 'edit', @http_endpoint
+      render_form_errors "edit", @http_endpoint
     end
   end
 

app/controllers/ip_addresses_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class IPAddressesController < ApplicationController
 
   before_action :admin_required
@@ -13,7 +15,7 @@ class IPAddressesController < ApplicationController
     if @ip_address.save
       redirect_to_with_json [:edit, @ip_pool]
     else
-      render_form_errors 'new', @ip_address
+      render_form_errors "new", @ip_address
     end
   end
 
@@ -21,7 +23,7 @@ class IPAddressesController < ApplicationController
     if @ip_address.update(safe_params)
       redirect_to_with_json [:edit, @ip_pool]
     else
-      render_form_errors 'edit', @ip_address
+      render_form_errors "edit", @ip_address
     end
   end
 

app/controllers/ip_pool_rules_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class IPPoolRulesController < ApplicationController
 
   include WithinOrganization
@@ -29,7 +31,7 @@ class IPPoolRulesController < ApplicationController
     if @ip_pool_rule.save
       redirect_to_with_json [organization, @server, :ip_pool_rules]
     else
-      render_form_errors 'new', @ip_pool_rule
+      render_form_errors "new", @ip_pool_rule
     end
   end
 
@@ -37,7 +39,7 @@ class IPPoolRulesController < ApplicationController
     if @ip_pool_rule.update(safe_params)
       redirect_to_with_json [organization, @server, :ip_pool_rules]
     else
-      render_form_errors 'edit', @ip_pool_rule
+      render_form_errors "edit", @ip_pool_rule
     end
   end
 

app/controllers/ip_pools_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class IPPoolsController < ApplicationController
 
   before_action :admin_required
@@ -14,25 +16,25 @@ class IPPoolsController < ApplicationController
   def create
     @ip_pool = IPPool.new(safe_params)
     if @ip_pool.save
-      redirect_to_with_json [:edit, @ip_pool], :notice => "IP Pool has been added successfully. You can now add IP addresses to it."
+      redirect_to_with_json [:edit, @ip_pool], notice: "IP Pool has been added successfully. You can now add IP addresses to it."
     else
-      render_form_errors 'new', @ip_pool
+      render_form_errors "new", @ip_pool
     end
   end
 
   def update
     if @ip_pool.update(safe_params)
-      redirect_to_with_json [:edit, @ip_pool], :notice => "IP Pool has been updated."
+      redirect_to_with_json [:edit, @ip_pool], notice: "IP Pool has been updated."
     else
-      render_form_errors 'edit', @ip_pool
+      render_form_errors "edit", @ip_pool
     end
   end
 
   def destroy
     @ip_pool.destroy
-    redirect_to_with_json :ip_pools, :notice => "IP pool has been removed successfully."
-  rescue ActiveRecord::DeleteRestrictionError => e
-    redirect_to_with_json [:edit, @ip_pool], :alert => "IP pool cannot be removed because it is still assigned to servers/rules."
+    redirect_to_with_json :ip_pools, notice: "IP pool has been removed successfully."
+  rescue ActiveRecord::DeleteRestrictionError
+    redirect_to_with_json [:edit, @ip_pool], alert: "IP pool cannot be removed because it still has associated addresses or servers."
   end
 
   private

app/controllers/legacy_api/base_controller.rb

@@ -0,0 +1,133 @@
+# frozen_string_literal: true
+
+module LegacyAPI
+  # The Legacy API is the Postal v1 API which existed from the start with main
+  # aim of allowing e-mails to sent over HTTP rather than SMTP. The API itself
+  # did not feature much functionality. This API was implemented using Moonrope
+  # which was a self documenting API tool, however, is now no longer maintained.
+  # In light of that, these controllers now implement the same functionality as
+  # the original Moonrope API without the actual requirement to use any of the
+  # Moonrope components.
+  #
+  # Important things to note about the API:
+  #
+  #   * Moonrope allow params to be provided as JSON in the body of the request
+  #     along with the application/json content type. It also allowed for params
+  #     to be sent in the 'params' parameter when using the
+  #     application/x-www-form-urlencoded content type. Both methods are supported.
+  #
+  #   * Authentication is performed using a X-Server-API-Key variable.
+  #
+  #   * The method used to make the request is not important. Most clients use POST
+  #     but other methods should be supported. The routing for this legacvy
+  #     API supports GET, POST, PUT and PATCH.
+  #
+  #   * The status code for responses will always be 200 OK. The actual status of
+  #     a request is determined by the value of the 'status' attribute in the
+  #     returned JSON.
+  class BaseController < ActionController::Base
+
+    skip_before_action :set_browser_id
+    skip_before_action :verify_authenticity_token
+
+    before_action :start_timer
+    before_action :authenticate_as_server
+
+    private
+
+    # The Moonrope API spec allows for parameters to be provided in the body
+    # along with the application/json content type or they can be provided,
+    # as JSON, in the 'params' parameter when used with the
+    # application/x-www-form-urlencoded content type. This legacy API needs
+    # support both options for maximum compatibility.
+    #
+    # @return [Hash]
+    def api_params
+      if request.headers["content-type"] =~ /\Aapplication\/json/
+        return params.to_unsafe_hash
+      end
+
+      if params["params"].present?
+        return JSON.parse(params["params"])
+      end
+
+      {}
+    end
+
+    # The API returns a length of time to complete a request. We'll start
+    # a timer when the request starts and then use this method to calculate
+    # the time taken to complete the request.
+    #
+    # @return [void]
+    def start_timer
+      @start_time = Time.now.to_f
+    end
+
+    # The only method available to authenticate to the legacy API is using a
+    # credential from the server itself. This method will attempt to find
+    # that credential from the X-Server-API-Key header and will set the
+    # current_credential instance variable if a token is valid. Otherwise it
+    # will render an error to halt execution.
+    #
+    # @return [void]
+    def authenticate_as_server
+      key = request.headers["X-Server-API-Key"]
+      if key.blank?
+        render_error "AccessDenied",
+                     message: "Must be authenticated as a server."
+        return
+      end
+
+      credential = Credential.where(type: "API", key: key).first
+      if credential.nil?
+        render_error "InvalidServerAPIKey",
+                     message: "The API token provided in X-Server-API-Key was not valid.",
+                     token: key
+        return
+      end
+
+      if credential.server.suspended?
+        render_error "ServerSuspended"
+        return
+      end
+
+      credential.use
+      @current_credential = credential
+    end
+
+    # Render a successful response to the client
+    #
+    # @param [Hash] data
+    # @return [void]
+    def render_success(data)
+      render json: { status: "success",
+                     time: (Time.now.to_f - @start_time).round(3),
+                     flags: {},
+                     data: data }
+    end
+
+    # Render an error response to the client
+    #
+    # @param [String] code
+    # @param [Hash] data
+    # @return [void]
+    def render_error(code, data = {})
+      render json: { status: "error",
+                     time: (Time.now.to_f - @start_time).round(3),
+                     flags: {},
+                     data: data.merge(code: code) }
+    end
+
+    # Render a parameter error response to the client
+    #
+    # @param [String] message
+    # @return [void]
+    def render_parameter_error(message)
+      render json: { status: "parameter-error",
+                     time: (Time.now.to_f - @start_time).round(3),
+                     flags: {},
+                     data: { message: message } }
+    end
+
+  end
+end

app/controllers/legacy_api/messages_controller.rb

@@ -0,0 +1,166 @@
+# frozen_string_literal: true
+
+module LegacyAPI
+  class MessagesController < BaseController
+
+    # Returns details about a given message
+    #
+    #   URL:            /api/v1/messages/message
+    #
+    #   Parameters:     id              => REQ: The ID of the message
+    #                   _expansions     => An array of types of details t
+    #                                      to return
+    #
+    #   Response:       A hash containing message information
+    #                   OR an error if the message does not exist.
+    #
+    def message
+      message = find_message
+      return if performed?
+
+      message_hash = { id: message.id, token: message.token }
+      expansions = api_params["_expansions"]
+
+      if expansions == true || (expansions.is_a?(Array) && expansions.include?("status"))
+        message_hash[:status] = {
+          status: message.status,
+          last_delivery_attempt: message.last_delivery_attempt&.to_f,
+          held: message.held,
+          hold_expiry: message.hold_expiry&.to_f
+        }
+      end
+
+      if expansions == true || (expansions.is_a?(Array) && expansions.include?("details"))
+        message_hash[:details] = {
+          rcpt_to: message.rcpt_to,
+          mail_from: message.mail_from,
+          subject: message.subject,
+          message_id: message.message_id,
+          timestamp: message.timestamp.to_f,
+          direction: message.scope,
+          size: message.size,
+          bounce: message.bounce,
+          bounce_for_id: message.bounce_for_id,
+          tag: message.tag,
+          received_with_ssl: message.received_with_ssl
+        }
+      end
+
+      if expansions == true || (expansions.is_a?(Array) && expansions.include?("inspection"))
+        message_hash[:inspection] = {
+          inspected: message.inspected,
+          spam: message.spam,
+          spam_score: message.spam_score.to_f,
+          threat: message.threat,
+          threat_details: message.threat_details
+        }
+      end
+
+      if expansions == true || (expansions.is_a?(Array) && expansions.include?("plain_body"))
+        message_hash[:plain_body] = message.plain_body
+      end
+
+      if expansions == true || (expansions.is_a?(Array) && expansions.include?("html_body"))
+        message_hash[:html_body] = message.html_body
+      end
+
+      if expansions == true || (expansions.is_a?(Array) && expansions.include?("attachments"))
+        message_hash[:attachments] = message.attachments.map do |attachment|
+          {
+            filename: attachment.filename.to_s,
+            content_type: attachment.mime_type,
+            data: Base64.encode64(attachment.body.to_s),
+            size: attachment.body.to_s.bytesize,
+            hash: Digest::SHA1.hexdigest(attachment.body.to_s)
+          }
+        end
+      end
+
+      if expansions == true || (expansions.is_a?(Array) && expansions.include?("headers"))
+        message_hash[:headers] = message.headers
+      end
+
+      if expansions == true || (expansions.is_a?(Array) && expansions.include?("raw_message"))
+        message_hash[:raw_message] = Base64.encode64(message.raw_message)
+      end
+
+      if expansions == true || (expansions.is_a?(Array) && expansions.include?("activity_entries"))
+        message_hash[:activity_entries] = {
+          loads: message.loads,
+          clicks: message.clicks
+        }
+      end
+
+      render_success message_hash
+    rescue Postal::MessageDB::Message::NotFound
+      render_error "MessageNotFound",
+                   message: "No message found matching provided ID",
+                   id: api_params["id"]
+    end
+
+    # Returns all the deliveries for a given message
+    #
+    #   URL:            /api/v1/messages/deliveries
+    #
+    #   Parameters:     id              => REQ: The ID of the message
+    #
+    #   Response:       A array of hashes containing delivery information
+    #                   OR an error if the message does not exist.
+    #
+    def deliveries
+      message = find_message
+      return if performed?
+
+      deliveries = message.deliveries.map do |d|
+        {
+          id: d.id,
+          status: d.status,
+          details: d.details,
+          output: d.output&.strip,
+          sent_with_ssl: d.sent_with_ssl,
+          log_id: d.log_id,
+          time: d.time&.to_f,
+          timestamp: d.timestamp.to_f
+        }
+      end
+      render_success deliveries
+    rescue Postal::MessageDB::Message::NotFound
+      render_error "MessageNotFound",
+                   message: "No message found matching provided ID",
+                   id: api_params["id"]
+    end
+
+    private
+
+    # Look up the message referenced by the request's `id` parameter.
+    #
+    # The legacy API only ever identifies a message by its integer ID. The
+    # request body is parsed as JSON, so without validation a JSON object or
+    # array supplied for `id` would arrive as a Ruby Hash/Array and be passed
+    # straight through to the message database as a raw set of SQL conditions.
+    # We therefore reject anything that is not a simple scalar before it can
+    # reach the database and coerce the value to an integer ID.
+    #
+    # Renders an error and returns nil when the parameter is missing or is not
+    # a scalar; otherwise returns the matched message (raising NotFound when no
+    # message matches, which the actions rescue).
+    #
+    # @return [Postal::MessageDB::Message, nil]
+    def find_message
+      id = api_params["id"]
+
+      if id.blank?
+        render_parameter_error "`id` parameter is required but is missing"
+        return
+      end
+
+      unless id.is_a?(String) || id.is_a?(Integer)
+        render_parameter_error "`id` parameter must be a string or integer"
+        return
+      end
+
+      @current_credential.server.message(id.to_i)
+    end
+
+  end
+end

app/controllers/legacy_api/send_controller.rb

@@ -0,0 +1,135 @@
+# frozen_string_literal: true
+
+module LegacyAPI
+  class SendController < BaseController
+
+    ERROR_MESSAGES = {
+      "NoRecipients" => "There are no recipients defined to receive this message",
+      "NoContent" => "There is no content defined for this e-mail",
+      "TooManyToAddresses" => "The maximum number of To addresses has been reached (maximum 50)",
+      "TooManyCCAddresses" => "The maximum number of CC addresses has been reached (maximum 50)",
+      "TooManyBCCAddresses" => "The maximum number of BCC addresses has been reached (maximum 50)",
+      "FromAddressMissing" => "The From address is missing and is required",
+      "UnauthenticatedFromAddress" => "The From address is not authorised to send mail from this server",
+      "AttachmentMissingName" => "An attachment is missing a name",
+      "AttachmentMissingData" => "An attachment is missing data"
+    }.freeze
+
+    # Send a message with the given options
+    #
+    #   URL:            /api/v1/send/message
+    #
+    #   Parameters:     to              => REQ: An array of emails addresses
+    #                   cc              => An array of email addresses to CC
+    #                   bcc             => An array of email addresses to BCC
+    #                   from            => The name/email to send the email from
+    #                   sender          => The name/email of the 'Sender'
+    #                   reply_to        => The name/email of the 'Reply-to'
+    #                   plain_body      => The plain body
+    #                   html_body       => The HTML body
+    #                   bounce          => Is this message a bounce?
+    #                   tag             => A custom tag to add to the message
+    #                   custom_headers  => A hash of custom headers
+    #                   attachments     => An array of attachments
+    #                                      (name, content_type and data (base64))
+    #
+    #   Response:       A array of hashes containing message information
+    #                   OR an error if there is an issue sending the message
+    #
+    def message
+      attributes = {}
+      attributes[:to] = api_params["to"]
+      attributes[:cc] = api_params["cc"]
+      attributes[:bcc] = api_params["bcc"]
+      attributes[:from] = api_params["from"]
+      attributes[:sender] = api_params["sender"]
+      attributes[:subject] = api_params["subject"]
+      attributes[:reply_to] = api_params["reply_to"]
+      attributes[:plain_body] = api_params["plain_body"]
+      attributes[:html_body] = api_params["html_body"]
+      attributes[:bounce] = api_params["bounce"] ? true : false
+      attributes[:tag] = api_params["tag"]
+      attributes[:custom_headers] = api_params["headers"] if api_params["headers"]
+      attributes[:attachments] = []
+
+      (api_params["attachments"] || []).each do |attachment|
+        next unless attachment.is_a?(Hash)
+
+        attributes[:attachments] << { name: attachment["name"], content_type: attachment["content_type"], data: attachment["data"], base64: true }
+      end
+
+      message = OutgoingMessagePrototype.new(@current_credential.server, request.ip, "api", attributes)
+      message.credential = @current_credential
+      if message.valid?
+        result = message.create_messages
+        render_success message_id: message.message_id, messages: result
+      else
+        render_error message.errors.first, message: ERROR_MESSAGES[message.errors.first]
+      end
+    end
+
+    # Send a message by providing a raw message
+    #
+    #   URL:            /api/v1/send/raw
+    #
+    #   Parameters:     rcpt_to         => REQ: An array of email addresses to send
+    #                                      the message to
+    #                   mail_from       => REQ: the address to send the email from
+    #                   data            => REQ: base64-encoded mail data
+    #
+    #   Response:       A array of hashes containing message information
+    #                   OR an error if there is an issue sending the message
+    #
+    def raw
+      unless api_params["rcpt_to"].is_a?(Array)
+        render_parameter_error "`rcpt_to` parameter is required but is missing"
+        return
+      end
+
+      if api_params["mail_from"].blank?
+        render_parameter_error "`mail_from` parameter is required but is missing"
+        return
+      end
+
+      if api_params["data"].blank?
+        render_parameter_error "`data` parameter is required but is missing"
+        return
+      end
+
+      # Decode the raw message
+      raw_message = Base64.decode64(api_params["data"])
+
+      # Parse through mail to get the from/sender headers
+      mail = Mail.new(raw_message.split("\r\n\r\n", 2).first)
+      from_headers = { "from" => mail.from, "sender" => mail.sender }
+      authenticated_domain = @current_credential.server.find_authenticated_domain_from_headers(from_headers)
+
+      # If we're not authenticated, don't continue
+      if authenticated_domain.nil?
+        render_error "UnauthenticatedFromAddress"
+        return
+      end
+
+      # Store the result ready to return
+      result = { message_id: nil, messages: {} }
+      if api_params["rcpt_to"].is_a?(Array)
+        api_params["rcpt_to"].uniq.each do |rcpt_to|
+          message = @current_credential.server.message_db.new_message
+          message.rcpt_to = rcpt_to
+          message.mail_from = api_params["mail_from"]
+          message.raw_message = raw_message
+          message.received_with_ssl = true
+          message.scope = "outgoing"
+          message.domain_id = authenticated_domain.id
+          message.credential_id = @current_credential.id
+          message.bounce = api_params["bounce"] ? true : false
+          message.save
+          result[:message_id] = message.message_id if result[:message_id].nil?
+          result[:messages][rcpt_to] = { id: message.id, token: message.token }
+        end
+      end
+      render_success result
+    end
+
+  end
+end

app/controllers/messages_controller.rb

@@ -1,4 +1,6 @@
-  class MessagesController < ApplicationController
+# frozen_string_literal: true
+
+class MessagesController < ApplicationController
 
   include WithinOrganization
 
@@ -6,44 +8,44 @@
   before_action { params[:id] && @message = @server.message_db.message(params[:id].to_i) }
 
   def new
-    if params[:direction] == 'incoming'
-      @message = IncomingMessagePrototype.new(@server, request.ip, 'web-ui', {})
+    if params[:direction] == "incoming"
+      @message = IncomingMessagePrototype.new(@server, request.ip, "web-ui", {})
       @message.from = session[:test_in_from] || current_user.email_tag
       @message.to = @server.routes.order(:name).first&.description
     else
-      @message = OutgoingMessagePrototype.new(@server, request.ip, 'web-ui', {})
+      @message = OutgoingMessagePrototype.new(@server, request.ip, "web-ui", {})
       @message.to = session[:test_out_to] || current_user.email_address
       if domain = @server.domains.verified.order(:name).first
         @message.from = "test@#{domain.name}"
       end
     end
-    @message.subject = "Test Message at #{Time.zone.now.to_s(:long)}"
+    @message.subject = "Test Message at #{Time.zone.now.to_fs(:long)}"
     @message.plain_body = "This is a message to test the delivery of messages through Postal."
   end
 
   def create
-    if params[:direction] == 'incoming'
+    if params[:direction] == "incoming"
       session[:test_in_from] = params[:message][:from] if params[:message]
-      @message = IncomingMessagePrototype.new(@server, request.ip, 'web-ui', params[:message])
-      @message.attachments = [{:name => "test.txt", :content_type => "text/plain", :data => "Hello world!"}]
+      @message = IncomingMessagePrototype.new(@server, request.ip, "web-ui", params[:message])
+      @message.attachments = [{ name: "test.txt", content_type: "text/plain", data: "Hello world!" }]
     else
       session[:test_out_to] = params[:message][:to] if params[:message]
-      @message = OutgoingMessagePrototype.new(@server, request.ip, 'web-ui', params[:message])
+      @message = OutgoingMessagePrototype.new(@server, request.ip, "web-ui", params[:message])
     end
     if result = @message.create_messages
       if result.size == 1
-        redirect_to_with_json organization_server_message_path(organization, @server, result.first.last[:id]), :notice => "Message was queued successfully"
+        redirect_to_with_json organization_server_message_path(organization, @server, result.first.last[:id]), notice: "Message was queued successfully"
       else
-        redirect_to_with_json [:queue, organization, @server], :notice => "Messages queued successfully "
+        redirect_to_with_json [:queue, organization, @server], notice: "Messages queued successfully "
       end
     else
       respond_to do |wants|
         wants.html do
           flash.now[:alert] = "Your message could not be sent. Ensure that all fields are completed fully. #{result.errors.inspect}"
-          render 'new'
+          render "new"
         end
         wants.json do
-          render :json => {:flash => {:alert => "Your message could not be sent. Please check all field are completed fully."}}
+          render json: { flash: { alert: "Your message could not be sent. Please check all field are completed fully." } }
         end
       end
 
@@ -52,71 +54,87 @@
 
   def outgoing
     @searchable = true
-    get_messages('outgoing')
+    get_messages("outgoing")
     respond_to do |wants|
       wants.html
-      wants.json { render :json => {
-        :flash => flash.each_with_object({}) { |(type, message), hash| hash[type] = message},
-        :region_html => render_to_string(:partial => 'index', :formats => [:html])
-      }}
+      wants.json do
+        render json: {
+          flash: flash.each_with_object({}) { |(type, message), hash| hash[type] = message },
+          region_html: render_to_string(partial: "index", formats: [:html])
+        }
+      end
     end
   end
 
   def incoming
     @searchable = true
-    get_messages('incoming')
+    get_messages("incoming")
     respond_to do |wants|
       wants.html
-      wants.json { render :json => {
-        :flash => flash.each_with_object({}) { |(type, message), hash| hash[type] = message},
-        :region_html => render_to_string(:partial => 'index', :formats => [:html])
-      }}
+      wants.json do
+        render json: {
+          flash: flash.each_with_object({}) { |(type, message), hash| hash[type] = message },
+          region_html: render_to_string(partial: "index", formats: [:html])
+        }
+      end
     end
   end
 
   def held
-    get_messages('held')
+    get_messages("held")
   end
 
   def deliveries
-    render :json => {:html => render_to_string(:partial => 'deliveries', :locals => {:message => @message})}
+    render json: { html: render_to_string(partial: "deliveries", locals: { message: @message }) }
   end
 
   def html_raw
-    render :html => @message.html_body_without_tracking_image.html_safe
+    override_content_security_policy_directives(
+      default_src: %w('none'),
+      script_src: %w('none'),
+      style_src: %w('unsafe-inline'),
+      img_src: %w(* data:),
+      font_src: %w(*),
+      frame_ancestors: %w('self'),
+      form_action: %w('none'),
+      base_uri: %w('none')
+    )
+    response.headers["X-Content-Type-Options"] = "nosniff"
+    response.headers["Referrer-Policy"] = "no-referrer"
+    render html: @message.html_body_without_tracking_image.html_safe
   end
 
   def spam_checks
-    @spam_checks = @message.spam_checks.sort_by { |s| s['score']}.reverse
+    @spam_checks = @message.spam_checks.sort_by { |s| s["score"] }.reverse
   end
 
   def attachment
     if @message.attachments.size > params[:attachment].to_i
       attachment = @message.attachments[params[:attachment].to_i]
-      send_data attachment.body, :content_type => attachment.mime_type, :disposition => 'download', :filename => attachment.filename
+      send_data attachment.body, content_type: attachment.mime_type, disposition: "download", filename: attachment.filename
     else
-      redirect_to attachments_organization_server_message_path(organization, @server, @message.id), :alert => "Attachment not found. Choose an attachment from the list below."
+      redirect_to attachments_organization_server_message_path(organization, @server, @message.id), alert: "Attachment not found. Choose an attachment from the list below."
     end
   end
 
   def download
     if @message.raw_message
-      send_data @message.raw_message, :filename => "Message-#{organization.permalink}-#{@server.permalink}-#{@message.id}.eml", :content_type => "text/plain"
+      send_data @message.raw_message, filename: "Message-#{organization.permalink}-#{@server.permalink}-#{@message.id}.eml", content_type: "text/plain"
     else
-      redirect_to organization_server_message_path(organization, @server, @message.id), :alert => "We no longer have the raw message stored for this message."
+      redirect_to organization_server_message_path(organization, @server, @message.id), alert: "We no longer have the raw message stored for this message."
     end
   end
 
   def retry
     if @message.raw_message?
       if @message.queued_message
-        @message.queued_message.queue!
+        @message.queued_message.retry_now
         flash[:notice] = "This message will be retried shortly."
       elsif @message.held?
-        @message.add_to_message_queue(:manual => true)
+        @message.add_to_message_queue(manual: true)
         flash[:notice] = "This message has been released. Delivery will be attempted shortly."
       else
-        @message.add_to_message_queue(:manual => true)
+        @message.add_to_message_queue(manual: true)
         flash[:notice] = "This message will be redelivered shortly."
       end
     else
@@ -148,20 +166,20 @@
   private
 
   def get_messages(scope)
-    if scope == 'held'
-      options = {:where => {:held => 1}}
+    if scope == "held"
+      options = { where: { held: true } }
     else
-      options = {:where => {:scope => scope, :spam => false}, :order => :timestamp, :direction => 'desc'}
+      options = { where: { scope: scope, spam: false }, order: :timestamp, direction: "desc" }
 
       if @query = (params[:query] || session["msg_query_#{@server.id}_#{scope}"]).presence
         session["msg_query_#{@server.id}_#{scope}"] = @query
-        qs = Postal::QueryString.new(@query)
+        qs = QueryString.new(@query)
         if qs.empty?
           flash.now[:alert] = "It doesn't appear you entered anything to filter on. Please double check your query."
         else
           @queried = true
-          if qs[:order] == 'oldest-first'
-            options[:direction] = 'asc'
+          if qs[:order] == "oldest-first"
+            options[:direction] = "asc"
           end
 
           options[:where][:rcpt_to] = qs[:to] if qs[:to]
@@ -176,13 +194,13 @@
           end
           options[:where][:tag] = qs[:tag] if qs[:tag]
           options[:where][:id] = qs[:id] if qs[:id]
-          options[:where][:spam] = true if qs[:spam] == 'yes' || qs[:spam] == 'y'
+          options[:where][:spam] = true if qs[:spam] == "yes" || qs[:spam] == "y"
           if qs[:before] || qs[:after]
             options[:where][:timestamp] = {}
             if qs[:before]
               begin
                 options[:where][:timestamp][:less_than] = get_time_from_string(qs[:before]).to_f
-              rescue TimeUndetermined => e
+              rescue TimeUndetermined
                 flash.now[:alert] = "Couldn't determine time for before from '#{qs[:before]}'"
               end
             end
@@ -190,7 +208,7 @@
             if qs[:after]
               begin
                 options[:where][:timestamp][:greater_than] = get_time_from_string(qs[:after]).to_f
-              rescue TimeUndetermined => e
+              rescue TimeUndetermined
                 flash.now[:alert] = "Couldn't determine time for after from '#{qs[:after]}'"
               end
             end
@@ -208,21 +226,20 @@
 
   def get_time_from_string(string)
     begin
-      if string =~ /\A(\d{2,4})\-(\d{2})\-(\d{2}) (\d{2})\:(\d{2})\z/
-        time = Time.new($1.to_i, $2.to_i, $3.to_i, $4.to_i, $5.to_i)
-      elsif string =~ /\A(\d{2,4})\-(\d{2})\-(\d{2})\z/
-        time = Time.new($1.to_i, $2.to_i, $3.to_i, 0)
+      if string =~ /\A(\d{2,4})-(\d{2})-(\d{2}) (\d{2}):(\d{2})\z/
+        time = Time.new(::Regexp.last_match(1).to_i, ::Regexp.last_match(2).to_i, ::Regexp.last_match(3).to_i, ::Regexp.last_match(4).to_i, ::Regexp.last_match(5).to_i)
+      elsif string =~ /\A(\d{2,4})-(\d{2})-(\d{2})\z/
+        time = Time.new(::Regexp.last_match(1).to_i, ::Regexp.last_match(2).to_i, ::Regexp.last_match(3).to_i, 0)
       else
-        time = Chronic.parse(string, :context => :past)
+        time = Chronic.parse(string, context: :past)
       end
-    rescue
+    rescue StandardError
+      time = nil
     end
 
-    if time.nil?
-      raise TimeUndetermined, "Couldn't determine a suitable time from '#{string}'"
-    else
-      time
-    end
+    raise TimeUndetermined, "Couldn't determine a suitable time from '#{string}'" if time.nil?
+
+    time
   end
 
 end

app/controllers/organization_ip_pools_controller.rb

@@ -1,7 +1,9 @@
+# frozen_string_literal: true
+
 class OrganizationIPPoolsController < ApplicationController
 
   include WithinOrganization
-  before_action :admin_required, :only => [:assignments]
+  before_action :admin_required, only: [:assignments]
 
   def index
     @ip_pools = organization.ip_pools.order(:name)
@@ -10,7 +12,7 @@ class OrganizationIPPoolsController < ApplicationController
   def assignments
     organization.ip_pool_ids = params[:ip_pools]
     organization.save!
-    redirect_to [organization, :ip_pools], :notice => "Organization IP pools have been updated successfully"
+    redirect_to [organization, :ip_pools], notice: "Organization IP pools have been updated successfully"
   end
 
 end

app/controllers/organizations_controller.rb

@@ -1,6 +1,8 @@
+# frozen_string_literal: true
+
 class OrganizationsController < ApplicationController
 
-  before_action :admin_required, :only => [:new, :create, :delete, :destroy]
+  before_action :admin_required, only: [:new, :create, :delete, :destroy]
 
   def index
     if current_user.admin?
@@ -17,48 +19,49 @@ class OrganizationsController < ApplicationController
     @organization = Organization.new
   end
 
+  def edit
+    @organization_obj = current_user.organizations_scope.find(organization.id)
+  end
+
   def create
     @organization = Organization.new(params.require(:organization).permit(:name, :permalink))
     @organization.owner = current_user
     if @organization.save
       redirect_to_with_json organization_root_path(@organization)
     else
-      render_form_errors 'new', @organization
+      render_form_errors "new", @organization
     end
   end
 
-  def edit
-    @organization_obj = current_user.organizations_scope.find(organization.id)
-  end
-
   def update
     @organization_obj = current_user.organizations_scope.find(organization.id)
     if @organization_obj.update(params.require(:organization).permit(:name, :time_zone))
-      redirect_to_with_json organization_settings_path(@organization_obj), :notice => "Settings for #{@organization_obj.name} have been saved successfully."
+      redirect_to_with_json organization_settings_path(@organization_obj), notice: "Settings for #{@organization_obj.name} have been saved successfully."
     else
-      render_form_errors 'edit', @organization_obj
+      render_form_errors "edit", @organization_obj
     end
   end
 
   def destroy
-    unless current_user.authenticate(params[:password])
+    if params[:confirm_text].blank? || params[:confirm_text].downcase.strip != organization.name.downcase.strip
       respond_to do |wants|
-        wants.html { redirect_to organization_delete_path(@organization), :alert => "The password you entered was not valid. Please check and try again." }
-        wants.json { render :json => {:alert => "The password you entered was invalid. Please check and try again."} }
+        alert_text = "The text you entered does not match the organization name. Please check and try again."
+        wants.html { redirect_to organization_delete_path(@organization), alert: alert_text }
+        wants.json { render json: { alert: alert_text } }
       end
       return
     end
 
     organization.soft_destroy
-    redirect_to_with_json root_path(:nrd => 1), :notice => "#{@organization.name} has been removed successfully."
+    redirect_to_with_json root_path(nrd: 1), notice: "#{@organization.name} has been removed successfully."
   end
 
   private
 
   def organization
-    if [:edit, :update, :delete, :destroy].include?(action_name.to_sym)
-      @organization ||= params[:org_permalink] ? current_user.organizations_scope.find_by_permalink!(params[:org_permalink]) : nil
-    end
+    return unless [:edit, :update, :delete, :destroy].include?(action_name.to_sym)
+
+    @organization ||= params[:org_permalink] ? current_user.organizations_scope.find_by_permalink!(params[:org_permalink]) : nil
   end
   helper_method :organization
 

app/controllers/routes_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class RoutesController < ApplicationController
 
   include WithinOrganization
@@ -18,7 +20,7 @@ class RoutesController < ApplicationController
     if @route.save
       redirect_to_with_json [organization, @server, :routes]
     else
-      render_form_errors 'new', @route
+      render_form_errors "new", @route
     end
   end
 
@@ -26,7 +28,7 @@ class RoutesController < ApplicationController
     if @route.update(safe_params)
       redirect_to_with_json [organization, @server, :routes]
     else
-      render_form_errors 'edit', @route
+      render_form_errors "edit", @route
     end
   end
 
@@ -38,7 +40,7 @@ class RoutesController < ApplicationController
   private
 
   def safe_params
-    params.require(:route).permit(:name, :domain_id, :spam_mode, :_endpoint, :additional_route_endpoints_array => [])
+    params.require(:route).permit(:name, :domain_id, :spam_mode, :_endpoint, additional_route_endpoints_array: [])
   end
 
 end

app/controllers/servers_controller.rb

@@ -1,8 +1,10 @@
+# frozen_string_literal: true
+
 class ServersController < ApplicationController
 
   include WithinOrganization
 
-  before_action :admin_required, :only => [:advanced, :suspend, :unsuspend]
+  before_action :admin_required, only: [:advanced, :suspend, :unsuspend]
   before_action { params[:id] && @server = organization.servers.present.find_by_permalink!(params[:id]) }
 
   def index
@@ -23,7 +25,7 @@ class ServersController < ApplicationController
     @first_date = graph_data.first.first
     @last_date = graph_data.last.first
     @graph_data = graph_data.map(&:last)
-    @messages = @server.message_db.messages(:order => 'id', :direction => 'desc', :limit => 6)
+    @messages = @server.message_db.messages(order: "id", direction: "desc", limit: 6)
   end
 
   def new
@@ -35,49 +37,60 @@ class ServersController < ApplicationController
     if @server.save
       redirect_to_with_json organization_server_path(organization, @server)
     else
-      render_form_errors 'new', @server
+      render_form_errors "new", @server
     end
   end
 
   def update
     extra_params = [:spam_threshold, :spam_failure_threshold, :postmaster_address]
-    extra_params += [:send_limit, :allow_sender, :log_smtp_data, :outbound_spam_threshold, :message_retention_days, :raw_message_retention_days, :raw_message_retention_size] if current_user.admin?
+
+    if current_user.admin?
+      extra_params += [
+        :send_limit,
+        :allow_sender,
+        :privacy_mode,
+        :log_smtp_data,
+        :outbound_spam_threshold,
+        :message_retention_days,
+        :raw_message_retention_days,
+        :raw_message_retention_size,
+      ]
+    end
+
     if @server.update(safe_params(*extra_params))
-      redirect_to_with_json organization_server_path(organization, @server), :notice => "Server settings have been updated"
+      redirect_to_with_json organization_server_path(organization, @server), notice: "Server settings have been updated"
     else
-      render_form_errors 'edit', @server
+      render_form_errors "edit", @server
     end
   end
 
   def destroy
-    unless current_user.authenticate(params[:password])
+    if params[:confirm_text].blank? || params[:confirm_text].downcase.strip != @server.name.downcase.strip
       respond_to do |wants|
-        wants.html do
-          redirect_to [:delete, organization, @server], :alert => "The password you entered was not valid. Please check and try again."
-        end
-        wants.json do
-          render :json => {:alert => "The password you entere was invalid. Please check and try again"}
-        end
+        alert_text = "The text you entered does not match the server name. Please check and try again."
+        wants.html { redirect_to organization_delete_path(@organization), alert: alert_text }
+        wants.json { render json: { alert: alert_text } }
       end
       return
     end
+
     @server.soft_destroy
-    redirect_to_with_json organization_root_path(organization), :notice => "#{@server.name} has been deleted successfully"
+    redirect_to_with_json organization_root_path(organization), notice: "#{@server.name} has been deleted successfully"
   end
 
   def queue
-    @messages = @server.queued_messages.order(:id => :desc).page(params[:page])
+    @messages = @server.queued_messages.order(id: :desc).page(params[:page]).includes(:ip_address)
     @messages_with_message = @messages.include_message
   end
 
   def suspend
     @server.suspend(params[:reason])
-    redirect_to_with_json [organization, @server], :notice => "Server has been suspended"
+    redirect_to_with_json [organization, @server], notice: "Server has been suspended"
   end
 
   def unsuspend
     @server.unsuspend
-    redirect_to_with_json [organization, @server], :notice => "Server has been unsuspended"
+    redirect_to_with_json [organization, @server], notice: "Server has been unsuspended"
   end
 
   private

app/controllers/sessions_controller.rb

@@ -1,28 +1,19 @@
+# frozen_string_literal: true
+
 class SessionsController < ApplicationController
 
-  layout 'sub'
+  layout "sub"
 
-  skip_before_action :login_required, :only => [:new, :create, :create_with_token, :begin_password_reset, :finish_password_reset, :ip, :raise_error]
+  before_action :require_local_authentication, only: [:create, :begin_password_reset, :finish_password_reset]
+  skip_before_action :login_required, only: [:new, :create, :begin_password_reset, :finish_password_reset, :ip, :raise_error, :create_from_oidc, :oauth_failure]
 
   def create
     login(User.authenticate(params[:email_address], params[:password]))
     flash[:remember_login] = true
     redirect_to_with_return_to root_path
-  rescue Postal::Errors::AuthenticationError => e
+  rescue Postal::Errors::AuthenticationError
     flash.now[:alert] = "The credentials you've provided are incorrect. Please check and try again."
-    render 'new'
-  end
-
-  def create_with_token
-    result = JWT.decode(params[:token], Postal.signing_key.to_s, 'HS256')[0]
-    if result['timestamp'] > 1.minute.ago.to_f
-      login(User.find(result['user'].to_i))
-      redirect_to root_path
-    else
-      destroy
-    end
-  rescue JWT::VerificationError
-    destroy
+    render "new"
   end
 
   def destroy
@@ -33,42 +24,76 @@ class SessionsController < ApplicationController
 
   def persist
     auth_session.persist! if logged_in?
-    render :plain => "OK"
+    render plain: "OK"
   end
 
   def begin_password_reset
-    if request.post?
-      if user = User.where(:email_address => params[:email_address]).first
-        user.begin_password_reset(params[:return_to])
-        redirect_to login_path(:return_to => params[:return_to]), :notice => "Please check your e-mail and click the link in the e-mail we've sent you."
-      else
-        redirect_to login_reset_path(:return_to => params[:return_to]), :alert => "No user exists with that e-mail address. Please check and try again."
-      end
+    return unless request.post?
+
+    user_scope = Postal::Config.oidc.enabled? ? User.with_password : User
+    user = user_scope.find_by(email_address: params[:email_address])
+
+    if user.nil?
+      redirect_to login_reset_path(return_to: params[:return_to]), alert: "No local user exists with that e-mail address. Please check and try again."
+      return
     end
+
+    user.begin_password_reset(params[:return_to])
+    redirect_to login_path(return_to: params[:return_to]), notice: "Please check your e-mail and click the link in the e-mail we've sent you."
   end
 
   def finish_password_reset
-    @user = User.where(:password_reset_token => params[:token]).where("password_reset_token_valid_until > ?", Time.now).first
+    @user = User.where(password_reset_token: params[:token]).where("password_reset_token_valid_until > ?", Time.now).first
     if @user.nil?
-      redirect_to login_path(:return_to => params[:return_to]), :alert => "This link has expired or never existed. Please choose reset password to try again."
+      redirect_to login_path(return_to: params[:return_to]), alert: "This link has expired or never existed. Please choose reset password to try again."
     end
 
-    if request.post?
-      if params[:password].blank?
-        flash.now[:alert] = "You must enter a new password"
-        return
-      end
-      @user.password = params[:password]
-      @user.password_confirmation = params[:password_confirmation]
-      if @user.save
-        login(@user)
-        redirect_to_with_return_to root_path, :notice => "Your new password has been set and you've been logged in."
-      end
+    return unless request.post?
+
+    if params[:password].blank?
+      flash.now[:alert] = "You must enter a new password"
+      return
     end
+
+    @user.password = params[:password]
+    @user.password_confirmation = params[:password_confirmation]
+    return unless @user.save
+
+    login(@user)
+    redirect_to_with_return_to root_path, notice: "Your new password has been set and you've been logged in."
   end
 
   def ip
-    render :plain => "ip: #{request.ip} remote ip: #{request.remote_ip}"
+    render plain: "ip: #{request.ip} remote ip: #{request.remote_ip}"
+  end
+
+  def create_from_oidc
+    unless Postal::Config.oidc.enabled?
+      raise Postal::Error, "OIDC cannot be used unless enabled in the configuration"
+    end
+
+    auth = request.env["omniauth.auth"]
+    user = User.find_from_oidc(auth.extra.raw_info, logger: Postal.logger)
+    if user.nil?
+      redirect_to login_path, alert: "No user was found matching your identity. Please contact your administrator."
+      return
+    end
+
+    login(user)
+    flash[:remember_login] = true
+    redirect_to_with_return_to root_path
+  end
+
+  def oauth_failure
+    redirect_to login_path, alert: "An issue occurred while logging you in with OpenID. Please try again later or contact your administrator."
+  end
+
+  private
+
+  def require_local_authentication
+    return if Postal::Config.oidc.local_authentication_enabled?
+
+    redirect_to login_path, alert: "Local authentication is not enabled"
   end
 
 end

app/controllers/smtp_endpoints_controller.rb

@@ -1,4 +1,7 @@
+# frozen_string_literal: true
+
 class SMTPEndpointsController < ApplicationController
+
   include WithinOrganization
   before_action { @server = organization.servers.present.find_by_permalink!(params[:server_id]) }
   before_action { params[:id] && @smtp_endpoint = @server.smtp_endpoints.find_by_uuid!(params[:id]) }
@@ -17,7 +20,7 @@ class SMTPEndpointsController < ApplicationController
       flash[:notice] = params[:return_notice] if params[:return_notice].present?
       redirect_to_with_json [:return_to, [organization, @server, :smtp_endpoints]]
     else
-      render_form_errors 'new', @smtp_endpoint
+      render_form_errors "new", @smtp_endpoint
     end
   end
 
@@ -25,7 +28,7 @@ class SMTPEndpointsController < ApplicationController
     if @smtp_endpoint.update(safe_params)
       redirect_to_with_json [organization, @server, :smtp_endpoints]
     else
-      render_form_errors 'edit', @smtp_endpoint
+      render_form_errors "edit", @smtp_endpoint
     end
   end
 

app/controllers/track_domains_controller.rb

@@ -1,4 +1,7 @@
+# frozen_string_literal: true
+
 class TrackDomainsController < ApplicationController
+
   include WithinOrganization
   before_action { @server = organization.servers.present.find_by_permalink!(params[:server_id]) }
   before_action { params[:id] && @track_domain = @server.track_domains.find_by_uuid!(params[:id]) }
@@ -16,7 +19,7 @@ class TrackDomainsController < ApplicationController
     if @track_domain.save
       redirect_to_with_json [:return_to, [organization, @server, :track_domains]]
     else
-      render_form_errors 'new', @track_domain
+      render_form_errors "new", @track_domain
     end
   end
 
@@ -24,7 +27,7 @@ class TrackDomainsController < ApplicationController
     if @track_domain.update(params.require(:track_domain).permit(:track_loads, :track_clicks, :excluded_click_domains, :ssl_enabled))
       redirect_to_with_json [organization, @server, :track_domains]
     else
-      render_form_errors 'edit', @track_domain
+      render_form_errors "edit", @track_domain
     end
   end
 
@@ -35,15 +38,15 @@ class TrackDomainsController < ApplicationController
 
   def check
     if @track_domain.check_dns
-      redirect_to_with_json [organization, @server, :track_domains], :notice => "Your CNAME for #{@track_domain.full_name} looks good!"
+      redirect_to_with_json [organization, @server, :track_domains], notice: "Your CNAME for #{@track_domain.full_name} looks good!"
     else
-      redirect_to_with_json [organization, @server, :track_domains], :alert => "There seems to be something wrong with your DNS record. Check documentation for information."
+      redirect_to_with_json [organization, @server, :track_domains], alert: "There seems to be something wrong with your DNS record. Check documentation for information."
     end
   end
 
   def toggle_ssl
-    @track_domain.update(:ssl_enabled => !@track_domain.ssl_enabled)
-    redirect_to_with_json [organization, @server, :track_domains], :notice => "SSL settings for #{@track_domain.full_name} updated successfully."
+    @track_domain.update(ssl_enabled: !@track_domain.ssl_enabled)
+    redirect_to_with_json [organization, @server, :track_domains], notice: "SSL settings for #{@track_domain.full_name} updated successfully."
   end
 
 end

app/controllers/user_controller.rb

@@ -1,16 +1,22 @@
+# frozen_string_literal: true
+
 class UserController < ApplicationController
 
-  skip_before_action :login_required, :only => [:new, :create, :join]
+  skip_before_action :login_required, only: [:new, :create, :join]
 
   def new
-    @user_invite = UserInvite.active.find_by!(:uuid => params[:invite_token])
+    @user_invite = UserInvite.active.find_by!(uuid: params[:invite_token])
     @user = User.new
     @user.email_address = @user_invite.email_address
-    render :layout => 'sub'
+    render layout: "sub"
+  end
+
+  def edit
+    @user = User.find(current_user.id)
   end
 
   def create
-    @user_invite = UserInvite.active.find_by!(:uuid => params[:invite_token])
+    @user_invite = UserInvite.active.find_by!(uuid: params[:invite_token])
     @user = User.new(params.require(:user).permit(:first_name, :last_name, :email_address, :password, :password_confirmation))
     @user.email_verified_at = Time.now
     if @user.save
@@ -18,74 +24,38 @@ class UserController < ApplicationController
       self.current_user = @user
       redirect_to root_path
     else
-      render 'new', :layout => 'sub'
+      render "new", layout: "sub"
     end
   end
 
-  def join
-    if @invite = UserInvite.where(:uuid => params[:token]).where("expires_at > ?", Time.now).first
-      if logged_in?
-        if request.post?
-          @invite.accept(current_user)
-          redirect_to_with_json root_path(:nrd => 1), :notice => "Invitation has been accepted successfully. You now have access to this organization."
-        elsif request.delete?
-          @invite.reject
-          redirect_to_with_json root_path(:nrd => 1), :notice => "Invitation has been rejected successfully."
-        else
-          @organizations = @invite.organizations.order(:name).to_a
-        end
-      else
-        redirect_to new_signup_path(params[:token])
-      end
-    else
-      redirect_to_with_json root_path(:nrd => 1), :alert => "The invite URL you have has expired. Please ask the person who invited you to re-send your invitation."
-    end
-  end
-
-  def edit
-    @user = User.find(current_user.id)
-  end
-
   def update
     @user = User.find(current_user.id)
-    @user.attributes = params.require(:user).permit(:first_name, :last_name, :time_zone, :email_address, :password, :password_confirmation)
+    safe_params = [:first_name, :last_name, :time_zone, :email_address]
 
-    if @user.authenticate_with_previous_password_first(params[:password])
-      @password_correct = true
-    else
-      respond_to do |wants|
-        wants.html do
-          flash.now[:alert] = "The current password you have entered is incorrect. Please check and try again."
-          render 'edit'
-        end
-        wants.json do
-          render :json => {:alert => "The current password you've entered is incorrect. Please check and try again"}
+    if @user.password? && Postal::Config.oidc.local_authentication_enabled?
+      safe_params += [:password, :password_confirmation]
+      if @user.authenticate_with_previous_password_first(params[:password])
+        @password_correct = true
+      else
+        respond_to do |wants|
+          wants.html do
+            flash.now[:alert] = "The current password you have entered is incorrect. Please check and try again."
+            render "edit"
+          end
+          wants.json do
+            render json: { alert: "The current password you've entered is incorrect. Please check and try again" }
+          end
         end
+        return
       end
-      return
     end
 
-    email_changed = @user.email_address_changed?
+    @user.attributes = params.require(:user).permit(safe_params)
 
     if @user.save
-      if email_changed
-        redirect_to_with_json verify_path(:return_to => settings_path), :notice => "Your settings have been updated successfully. As you've changed, your e-mail address you'll need to verify it before you can continue."
-      else
-        redirect_to_with_json settings_path, :notice => "Your settings have been updated successfully."
-      end
+      redirect_to_with_json settings_path, notice: "Your settings have been updated successfully."
     else
-      render_form_errors 'edit', @user
-    end
-  end
-
-  def verify
-    if request.post?
-      if params[:code].to_s.strip == current_user.email_verification_token.to_s || (Rails.env.development? && params[:code].to_s.strip == "123456")
-        current_user.verify!
-        redirect_to_with_json [:return_to, root_path], :notice => "Thanks - your e-mail address has been verified successfully."
-      else
-        flash_now :alert, "The code you've entered isn't correct. Please check and try again."
-      end
+      render_form_errors "edit", @user
     end
   end
 

app/controllers/users_controller.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class UsersController < ApplicationController
 
   before_action :admin_required
@@ -11,45 +13,44 @@ class UsersController < ApplicationController
     @user = User.new(admin: true)
   end
 
+  def edit
+  end
+
   def create
     @user = User.new(params.require(:user).permit(:email_address, :first_name, :last_name, :password, :password_confirmation, :admin, organization_ids: []))
     if @user.save
-      redirect_to_with_json :users, :notice => "#{@user.name} has been created successfully."
+      redirect_to_with_json :users, notice: "#{@user.name} has been created successfully."
     else
-      render_form_errors 'new', @user
+      render_form_errors "new", @user
     end
   end
 
-  def edit
-  end
-
   def update
     @user.attributes = params.require(:user).permit(:email_address, :first_name, :last_name, :admin, organization_ids: [])
 
     if @user == current_user && !@user.admin?
       respond_to do |wants|
         wants.html { redirect_to users_path, alert: "You cannot change your own admin status" }
-        wants.json { render :json => {:form_errors => ["You cannot change your own admin status"]}, :status => 422 }
+        wants.json { render json: { form_errors: ["You cannot change your own admin status"] }, status: :unprocessable_entity }
       end
       return
     end
 
     if @user.save
-      redirect_to_with_json :users, :notice => "Permissions for #{@user.name} have been updated successfully."
+      redirect_to_with_json :users, notice: "Permissions for #{@user.name} have been updated successfully."
     else
-      render_form_errors 'edit', @user
+      render_form_errors "edit", @user
     end
   end
 
   def destroy
     if @user == current_user
-      redirect_to_with_json :users, :alert => "You cannot delete your own user."
+      redirect_to_with_json :users, alert: "You cannot delete your own user."
       return
     end
 
     @user.destroy!
-    redirect_to_with_json :users, :notice => "#{@user.name} has been removed"
+    redirect_to_with_json :users, notice: "#{@user.name} has been removed"
   end
 
-
 end

app/controllers/webhooks_controller.rb

@@ -1,4 +1,7 @@
+# frozen_string_literal: true
+
 class WebhooksController < ApplicationController
+
   include WithinOrganization
   before_action { @server = organization.servers.present.find_by_permalink!(params[:server_id]) }
   before_action { params[:id] && @webhook = @server.webhooks.find_by_uuid!(params[:id]) }
@@ -8,7 +11,7 @@ class WebhooksController < ApplicationController
   end
 
   def new
-    @webhook = @server.webhooks.build(:all_events => true)
+    @webhook = @server.webhooks.build(all_events: true)
   end
 
   def create
@@ -16,7 +19,7 @@ class WebhooksController < ApplicationController
     if @webhook.save
       redirect_to_with_json [organization, @server, :webhooks]
     else
-      render_form_errors 'new', @webhook
+      render_form_errors "new", @webhook
     end
   end
 
@@ -24,7 +27,7 @@ class WebhooksController < ApplicationController
     if @webhook.update(safe_params)
       redirect_to_with_json [organization, @server, :webhooks]
     else
-      render_form_errors 'edit', @webhook
+      render_form_errors "edit", @webhook
     end
   end
 
@@ -45,7 +48,7 @@ class WebhooksController < ApplicationController
   private
 
   def safe_params
-    params.require(:webhook).permit(:name, :url, :all_events, :enabled, :events => [])
+    params.require(:webhook).permit(:name, :url, :all_events, :enabled, events: [])
   end
 
 end

app/controllers/well_known_controller.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+class WellKnownController < ApplicationController
+
+  layout false
+
+  skip_before_action :set_browser_id
+  skip_before_action :login_required
+  skip_before_action :set_timezone
+
+  def jwks
+    render json: JWT::JWK::Set.new(Postal.signer.jwk).export.to_json
+  end
+
+end

app/helpers/application_helper.rb

@@ -1,9 +1,12 @@
+# frozen_string_literal: true
+
 module ApplicationHelper
 
   def format_delivery_details(server, text)
-    text.gsub!(/\<msg\:(\d+)\>/) do
-      id = $1.to_i
-      link_to("message ##{id}", organization_server_message_path(server.organization, server, id), :class => "u-link")
+    text = h(text)
+    text.gsub!(/<msg:(\d+)>/) do
+      id = ::Regexp.last_match(1).to_i
+      link_to("message ##{id}", organization_server_message_path(server.organization, server, id), class: "u-link")
     end
     text.html_safe
   end
@@ -28,9 +31,9 @@ module ApplicationHelper
       server_domains = server.domains.verified.order(:name)
       unless server_domains.empty?
         s << "<optgroup label='Server Domains'>"
-        for domain in server_domains
-          selected = domain == selected_domain ? "selected='selected'" : ''
-          s << "<option value='#{domain.id}' #{selected}>#{domain.name}</option>"
+        server_domains.each do |domain|
+          selected = domain == selected_domain ? "selected='selected'" : ""
+          s << "<option value='#{h(domain.id)}' #{selected}>#{h(domain.name)}</option>"
         end
         s << "</optgroup>"
       end
@@ -38,13 +41,12 @@ module ApplicationHelper
       organization_domains = server.organization.domains.verified.order(:name)
       unless organization_domains.empty?
         s << "<optgroup label='Organization Domains'>"
-        for domain in organization_domains
-          selected = domain == selected_domain ? "selected='selected'" : ''
-          s << "<option value='#{domain.id}' #{selected}>#{domain.name}</option>"
+        organization_domains.each do |domain|
+          selected = domain == selected_domain ? "selected='selected'" : ""
+          s << "<option value='#{h(domain.id)}' #{selected}>#{h(domain.name)}</option>"
         end
         s << "</optgroup>"
       end
-
     end.html_safe
   end
 
@@ -55,22 +57,21 @@ module ApplicationHelper
       http_endpoints = server.http_endpoints.order(:name).to_a
       if http_endpoints.present?
         s << "<optgroup label='HTTP Endpoints'>"
-        for endpoint in http_endpoints
+        http_endpoints.each do |endpoint|
           value = "#{endpoint.class}##{endpoint.uuid}"
-          selected = value == selected_value ? "selected='selected'" : ''
-          s << "<option value='#{value}' #{selected}>#{endpoint.description}</option>"
+          selected = value == selected_value ? "selected='selected'" : ""
+          s << "<option value='#{h(value)}' #{selected}>#{h(endpoint.description)}</option>"
         end
         s << "</optgroup>"
       end
 
-
       smtp_endpoints = server.smtp_endpoints.order(:name).to_a
       if smtp_endpoints.present?
         s << "<optgroup label='SMTP Endpoints'>"
-        for endpoint in smtp_endpoints
+        smtp_endpoints.each do |endpoint|
           value = "#{endpoint.class}##{endpoint.uuid}"
-          selected = value == selected_value ? "selected='selected'" : ''
-          s << "<option value='#{value}' #{selected}>#{endpoint.description}</option>"
+          selected = value == selected_value ? "selected='selected'" : ""
+          s << "<option value='#{h(value)}' #{selected}>#{h(endpoint.description)}</option>"
         end
         s << "</optgroup>"
       end
@@ -78,10 +79,10 @@ module ApplicationHelper
       address_endpoints = server.address_endpoints.order(:address).to_a
       if address_endpoints.present?
         s << "<optgroup label='Address Endpoints'>"
-        for endpoint in address_endpoints
+        address_endpoints.each do |endpoint|
           value = "#{endpoint.class}##{endpoint.uuid}"
-          selected = value == selected_value ? "selected='selected'" : ''
-          s << "<option value='#{value}' #{selected}>#{endpoint.address}</option>"
+          selected = value == selected_value ? "selected='selected'" : ""
+          s << "<option value='#{h(value)}' #{selected}>#{h(endpoint.address)}</option>"
         end
         s << "</optgroup>"
       end
@@ -89,15 +90,22 @@ module ApplicationHelper
       unless options[:other] == false
         s << "<optgroup label='Other Options'>"
         Route::MODES.each do |mode|
-          next if mode == 'Endpoint'
-          selected = (selected_value == mode ? "selected='selected'" : '')
+          next if mode == "Endpoint"
+
+          selected = (selected_value == mode ? "selected='selected'" : "")
           text = t("route_modes.#{mode.underscore}")
-          s << "<option value='#{mode}' #{selected}>#{text}</option>"
+          s << "<option value='#{h(mode)}' #{selected}>#{h(text)}</option>"
         end
         s << "</optgroup>"
       end
-
     end.html_safe
   end
 
+  def postal_version_string
+    string = Postal.version
+    string += " (#{Postal.branch})" if Postal.branch &&
+                                       Postal.branch != "main"
+    string
+  end
+
 end

app/helpers/error_messages_helper.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+module ErrorMessagesHelper
+
+  # Renders a record's validation errors.
+  #
+  # This replaces the helper of the same name from the dynamic_form gem, which
+  # is unmaintained and cannot be installed on Ruby 3.4. The markup matches what
+  # that gem produced, minus the heading and description which the
+  # errorExplanation styles have always hidden, and matches the structure
+  # ajax.coffee builds when it renders errors from a remote form.
+  def error_messages_for(record)
+    record = record.to_model if record.respond_to?(:to_model)
+    return "".html_safe if record.nil? || record.errors.empty?
+
+    items = safe_join(record.errors.full_messages.map { |message| tag.li(message) })
+    tag.div(tag.ul(items), id: "errorExplanation", class: "errorExplanation")
+  end
+
+end

app/jobs/action_deletion_job.rb

@@ -1,12 +0,0 @@
-class ActionDeletionJob < Postal::Job
-  def perform
-    object = params['type'].constantize.deleted.find_by_id(params['id'])
-    if object
-      log "Deleting #{params['type']}##{params['id']}"
-      object.destroy
-      log "Deleted #{params['type']}##{params['id']}"
-    else
-      log "Couldn't find deleted object #{params['type']}##{params['id']}"
-    end
-  end
-end

app/jobs/action_deletions_job.rb

@@ -1,13 +0,0 @@
-class ActionDeletionsJob < Postal::Job
-  def perform
-    Organization.deleted.each do |org|
-      log "Permanently removing organization #{org.id} (#{org.permalink})"
-      org.destroy
-    end
-
-    Server.deleted.each do |server|
-      log "Permanently removing server #{server.id} (#{server.full_permalink})"
-      server.destroy
-    end
-  end
-end

app/jobs/check_all_dns_job.rb

@@ -1,13 +0,0 @@
-class CheckAllDNSJob < Postal::Job
-  def perform
-    Domain.where.not(:dns_checked_at => nil).where("dns_checked_at <= ?", 1.hour.ago).each do |domain|
-      log "Checking DNS for domain: #{domain.name}"
-      domain.check_dns(:auto)
-    end
-
-    TrackDomain.where("dns_checked_at IS NULL OR dns_checked_at <= ?", 1.hour.ago).includes(:domain).each do |domain|
-      log "Checking DNS for track domain: #{domain.full_name}"
-      domain.check_dns
-    end
-  end
-end

app/jobs/cleanup_authie_sessions_job.rb

@@ -1,7 +0,0 @@
-require 'authie/session'
-
-class CleanupAuthieSessionsJob < Postal::Job
-  def perform
-    Authie::Session.cleanup
-  end
-end

app/jobs/expire_held_messages_job.rb

@@ -1,12 +0,0 @@
-class ExpireHeldMessagesJob < Postal::Job
-  def perform
-    Server.all.each do |server|
-      messages = server.message_db.messages(:where => {
-        :status => 'Held',
-        :hold_expiry => {:less_than => Time.now.to_f}
-      })
-
-      messages.each(&:cancel_hold)
-    end
-  end
-end

app/jobs/prune_suppression_lists_job.rb

@@ -1,8 +0,0 @@
-class PruneSuppressionListsJob < Postal::Job
-  def perform
-    Server.all.each do |s|
-      log "Pruning suppression lists for server #{s.id}"
-      s.message_db.suppression_list.prune
-    end
-  end
-end

app/jobs/prune_webhook_requests_job.rb

@@ -1,8 +0,0 @@
-class PruneWebhookRequestsJob < Postal::Job
-  def perform
-    Server.all.each do |s|
-      log "Pruning webhook requests for server #{s.id}"
-      s.message_db.webhooks.prune
-    end
-  end
-end

app/jobs/requeue_webhooks_job.rb

@@ -1,5 +0,0 @@
-class RequeueWebhooksJob < Postal::Job
-  def perform
-    WebhookRequest.requeue_all
-  end
-end

app/jobs/send_notifications_job.rb

@@ -1,5 +0,0 @@
-class SendNotificationsJob < Postal::Job
-  def perform
-    Server.send_send_limit_notifications
-  end
-end

app/jobs/send_webhook_job.rb

@@ -1,28 +0,0 @@
-class SendWebhookJob < Postal::Job
-
-  def perform
-    if server = Server.find(params['server_id'])
-      new_items = {}
-      if params['payload']
-        for key, value in params['payload']
-          if key.to_s =~ /\A\_(\w+)/
-            begin
-              new_items[$1] = server.message_db.message(value.to_i).webhook_hash
-            rescue Postal::MessageDB::Message::NotFound
-            end
-          end
-        end
-      end
-
-      new_items.each do |key, value|
-        params['payload'].delete("_#{key}")
-        params['payload'][key] = value
-      end
-
-      WebhookRequest.trigger(server, params['event'], params['payload'])
-    else
-      log "Couldn't find server with ID #{params['server_id']}"
-    end
-  end
-
-end

app/jobs/sleep_job.rb

@@ -1,5 +0,0 @@
-class SleepJob < Postal::Job
-  def perform
-    sleep 5
-  end
-end

app/jobs/tidy_raw_messages_job.rb

@@ -1,7 +0,0 @@
-class TidyRawMessagesJob < Postal::Job
-
-  def perform
-
-  end
-
-end

app/jobs/unqueue_message_job.rb

@@ -1,462 +0,0 @@
-class UnqueueMessageJob < Postal::Job
-  def perform
-    if original_message = QueuedMessage.find_by_id(params['id'])
-      if original_message.acquire_lock
-
-        log "Lock acquired for queued message #{original_message.id}"
-
-        begin
-          original_message.message
-        rescue Postal::MessageDB::Message::NotFound
-          log "Unqueue #{original_message.id} because backend message has been removed."
-          original_message.destroy
-          return
-        end
-
-        unless original_message.retriable?
-          log "Skipping because retry after isn't reached"
-          original_message.unlock
-          return
-        end
-
-        begin
-          other_messages = original_message.batchable_messages(100)
-          log "Found #{other_messages.size} associated messages to process at the same time (batch key: #{original_message.batch_key})"
-        rescue
-          original_message.unlock
-          raise
-        end
-
-        ([original_message] + other_messages).each do |queued_message|
-          log_prefix = "[#{queued_message.server_id}::#{queued_message.message_id} #{queued_message.id}]"
-          begin
-            log "#{log_prefix} Got queued message with exclusive lock"
-
-            begin
-              queued_message.message
-            rescue Postal::MessageDB::Message::NotFound
-              log "#{log_prefix} Unqueueing #{queued_message.id} because backend message has been removed"
-              queued_message.destroy
-              next
-            end
-
-            #
-            # If the server is suspended, hold all messages
-            #
-            if queued_message.server.suspended?
-              log "#{log_prefix} Server is suspended. Holding message."
-              queued_message.message.create_delivery('Held', :details => "Mail server has been suspended. No e-mails can be processed at present. Contact support for assistance.")
-              queued_message.destroy
-              next
-            end
-
-            # We might not be able to send this any more, check the attempts
-            if queued_message.attempts >= Postal.config.general.maximum_delivery_attempts
-              details = "Maximum number of delivery attempts (#{queued_message.attempts}) has been reached."
-              if queued_message.message.scope == 'incoming'
-                # Send bounces to incoming e-mails when they are hard failed
-                if bounce_id = queued_message.send_bounce
-                  details += " Bounce sent to sender (see message <msg:#{bounce_id}>)"
-                end
-              elsif queued_message.message.scope == 'outgoing'
-                # Add the recipient to the suppression list
-                if queued_message.server.message_db.suppression_list.add(:recipient, queued_message.message.rcpt_to, :reason => "too many soft fails")
-                  log "Added #{queued_message.message.rcpt_to} to suppression list because maximum attempts has been reached"
-                  details += " Added #{queued_message.message.rcpt_to} to suppression list because delivery has failed #{queued_message.attempts} times."
-                end
-              end
-              queued_message.message.create_delivery('HardFail', :details => details)
-              queued_message.destroy
-              log "#{log_prefix} Message has reached maximum number of attempts. Hard failing."
-              next
-            end
-
-            # If the raw message has been removed (removed by retention)
-            unless queued_message.message.raw_message?
-              log "#{log_prefix} Raw message has been removed. Not sending."
-              queued_message.message.create_delivery('HardFail', :details => "Raw message has been removed. Cannot send message.")
-              queued_message.destroy
-              next
-            end
-
-            #
-            # Handle Incoming Messages
-            #
-            if queued_message.message.scope == 'incoming'
-              #
-              # If this is a bounce, we need to handle it as such
-              #
-              if queued_message.message.bounce == 1
-                log "#{log_prefix} Message is a bounce"
-                original_messages = queued_message.message.original_messages
-                unless original_messages.empty?
-                  for original_message in queued_message.message.original_messages
-                    queued_message.message.update(:bounce_for_id => original_message.id, :domain_id => original_message.domain_id)
-                    queued_message.message.create_delivery('Processed', :details => "This has been detected as a bounce message for <msg:#{original_message.id}>.")
-                    original_message.bounce!(queued_message.message)
-                    log "#{log_prefix} Bounce linked with message #{original_message.id}"
-                  end
-                  queued_message.destroy
-                  next
-                end
-
-                # This message was sent to the return path but hasn't been matched
-                # to an original message. If we have a route for this, route it
-                # otherwise we'll drop at this point.
-                if queued_message.message.route_id.nil?
-                  log "#{log_prefix} No source messages found. Hard failing."
-                  queued_message.message.create_delivery('HardFail', :details => "This message was a bounce but we couldn't link it with any outgoing message and there was no route for it.")
-                  queued_message.destroy
-                  next
-                end
-              end
-
-              #
-              # Update live stats
-              #
-              queued_message.message.database.live_stats.increment(queued_message.message.scope)
-
-              #
-              # Inspect incoming messages
-              #
-              if queued_message.message.inspected == 0
-                log "#{log_prefix} Inspecting message"
-                queued_message.message.inspect_message
-                if queued_message.message.inspected == 1
-                  is_spam = queued_message.message.spam_score > queued_message.server.spam_threshold
-                  queued_message.message.update(:spam => 1) if is_spam
-                  queued_message.message.append_headers(
-                    "X-Postal-Spam: #{queued_message.message.spam == 1 ? 'yes' : 'no'}",
-                    "X-Postal-Spam-Threshold: #{queued_message.server.spam_threshold}",
-                    "X-Postal-Spam-Score: #{queued_message.message.spam_score}",
-                    "X-Postal-Threat: #{queued_message.message.threat == 1 ? 'yes' : 'no'}"
-                  )
-                  log "#{log_prefix} Message inspected successfully. Headers added."
-                end
-              end
-
-              #
-              # If this message has a SPAM score higher than is permitted
-              #
-              if queued_message.message.spam_score >= queued_message.server.spam_failure_threshold
-                log "#{log_prefix} Message has a spam score higher than the server's maxmimum. Hard failing."
-                queued_message.message.create_delivery('HardFail', :details => "Message's spam score is higher than the failure threshold for this server. Threshold is currently #{queued_message.server.spam_failure_threshold}.")
-                queued_message.destroy
-                next
-              end
-
-              # If the server is in development mode, hold it
-              if queued_message.server.mode == 'Development' && !queued_message.manual?
-                log "Server is in development mode so holding."
-                queued_message.message.create_delivery('Held', :details => "Server is in development mode.")
-                queued_message.destroy
-                log "#{log_prefix} Server is in development mode. Holding."
-                next
-              end
-
-              #
-              # Find out what sort of message we're supposed to be sending and dispatch this request over to
-              # the sender.
-              #
-              if route = queued_message.message.route
-
-                # If the route says we're holding quananteed mail and this is spam, we'll hold this
-                if route.spam_mode == 'Quarantine' && queued_message.message.spam == 1 && !queued_message.manual?
-                  queued_message.message.create_delivery('Held', :details => "Message placed into quarantine.")
-                  queued_message.destroy
-                  log "#{log_prefix} Route says to quarantine spam message. Holding."
-                  next
-                end
-
-                # If the route says we're holding quananteed mail and this is spam, we'll hold this
-                if route.spam_mode == 'Fail' && queued_message.message.spam == 1 && !queued_message.manual?
-                  queued_message.message.create_delivery('HardFail', :details => "Message is spam and the route specifies it should be failed.")
-                  queued_message.destroy
-                  log "#{log_prefix} Route says to fail spam message. Hard failing."
-                  next
-                end
-
-                #
-                # Messages that should be blindly accepted are blindly accepted
-                #
-                if route.mode == 'Accept'
-                  queued_message.message.create_delivery('Processed', :details => "Message has been accepted but not sent to any endpoints.")
-                  queued_message.destroy
-                  log "#{log_prefix} Route says to accept without endpoint. Marking as processed."
-                  next
-                end
-
-                #
-                # Messages that should be accepted and held should be held
-                #
-                if route.mode == 'Hold'
-                  log "#{log_prefix} Route says to hold message."
-                  if queued_message.manual?
-                    log "#{log_prefix} Message was queued manually. Marking as processed."
-                    queued_message.message.create_delivery('Processed', :details => "Message has been processed.")
-                  else
-                    log "#{log_prefix} Message was not queued manually. Holding."
-                    queued_message.message.create_delivery('Held', :details => "Message has been accepted but not sent to any endpoints.")
-                  end
-                  queued_message.destroy
-                  next
-                end
-
-                #
-                # Messages that should be bounced should be bounced (or rejected if they got this far)
-                #
-                if route.mode == 'Bounce' || route.mode == 'Reject'
-                  if id = queued_message.send_bounce
-                    queued_message.message.create_delivery('HardFail', :details => "Message has been bounced because the route asks for this. See message <msg:#{id}>")
-                    log "#{log_prefix} Route says to bounce. Hard failing and sent bounce (#{id})."
-                  end
-                  queued_message.destroy
-                  next
-                end
-
-                begin
-                  if @fixed_result
-                    result = @fixed_result
-                  else
-                    case queued_message.message.endpoint
-                    when SMTPEndpoint
-                      sender = cached_sender(Postal::SMTPSender, queued_message.message.recipient_domain, nil, :servers => [queued_message.message.endpoint])
-                    when HTTPEndpoint
-                      sender = cached_sender(Postal::HTTPSender, queued_message.message.endpoint)
-                    when AddressEndpoint
-                      sender = cached_sender(Postal::SMTPSender, queued_message.message.endpoint.domain, nil, :force_rcpt_to => queued_message.message.endpoint.address)
-                    else
-                      log "#{log_prefix} Invalid endpoint for route (#{queued_message.message.endpoint_type})"
-                      queued_message.message.create_delivery('HardFail', :details => "Invalid endpoint for route.")
-                      queued_message.destroy
-                      next
-                    end
-                    result = sender.send_message(queued_message.message)
-                    if result.connect_error
-                      @fixed_result = result
-                    end
-                  end
-                end
-
-                # Log the result
-                log_details = result.details
-                if result.type =='HardFail' && result.suppress_bounce
-                  # The delivery hard failed, but requested that no bounce be sent
-                  log "#{log_prefix} Suppressing bounce message after hard fail"
-                elsif result.type =='HardFail' && queued_message.message.send_bounces?
-                  # If the message is a hard fail, send a bounce message for this message.
-                  log "#{log_prefix} Sending a bounce because message hard failed"
-                  if bounce_id = queued_message.send_bounce
-                    log_details += ". " unless log_details =~ /\.\z/
-                    log_details += " Sent bounce message to sender (see message <msg:#{bounce_id}>)"
-                  end
-                end
-
-                queued_message.message.create_delivery(result.type, :details => log_details, :output => result.output&.strip, :sent_with_ssl => result.secure, :log_id => result.log_id, :time => result.time)
-
-                if result.retry
-                  log "#{log_prefix} Message requeued for trying later."
-                  queued_message.retry_later(result.retry.is_a?(Integer) ? result.retry : nil)
-                  queued_message.allocate_ip_address
-                  queued_message.update_column(:ip_address_id, queued_message.ip_address&.id)
-                else
-                  log "#{log_prefix} Message processing completed."
-                  queued_message.message.endpoint.mark_as_used
-                  queued_message.destroy
-                end
-              else
-                log "#{log_prefix} No route and/or endpoint available for processing. Hard failing."
-                queued_message.message.create_delivery('HardFail', :details => "Message does not have a route and/or endpoint available for delivery.")
-                queued_message.destroy
-                next
-              end
-            end
-
-            #
-            # Handle Outgoing Messages
-            #
-            if queued_message.message.scope == 'outgoing'
-              if queued_message.message.domain.nil?
-                log "#{log_prefix} Message has no domain. Hard failing."
-                queued_message.message.create_delivery('HardFail', :details => "Message's domain no longer exist")
-                queued_message.destroy
-                next
-              end
-
-              #
-              # If there's no to address, we can't do much. Fail it.
-              #
-              if queued_message.message.rcpt_to.blank?
-                log "#{log_prefix} Message has no to address. Hard failing."
-                queued_message.message.create_delivery('HardFail', :details => "Message doesn't have an RCPT to")
-                queued_message.destroy
-                next
-              end
-
-              #
-              # If the credentials for this message is marked as holding and this isn't manual, hold it
-              #
-              if !queued_message.manual? && queued_message.message.credential && queued_message.message.credential.hold?
-                log "#{log_prefix} Credential wants us to hold messages. Holding."
-                queued_message.message.create_delivery('Held', :details => "Credential is configured to hold all messages authenticated by it.")
-                queued_message.destroy
-                next
-              end
-
-              #
-              # If the recipient is on the suppression list and this isn't a manual queueing block sending
-              #
-              if !queued_message.manual? && sl = queued_message.server.message_db.suppression_list.get(:recipient, queued_message.message.rcpt_to)
-                log "#{log_prefix} Recipient is on the suppression list. Holding."
-                queued_message.message.create_delivery('Held', :details => "Recipient (#{queued_message.message.rcpt_to}) is on the suppression list (reason: #{sl['reason']})")
-                queued_message.destroy
-                next
-              end
-
-              # Extract a tag and add it to the message if one doesn't exist
-              if queued_message.message.tag.nil? && tag = queued_message.message.headers['x-postal-tag']
-                log "#{log_prefix} Added tag #{tag.last}"
-                queued_message.message.update(:tag => tag.last)
-              end
-
-              # Parse the content of the message as appropriate
-              if queued_message.message.should_parse?
-                log "#{log_prefix} Parsing message content as it hasn't been parsed before"
-                queued_message.message.parse_content
-              end
-
-              # Inspect outgoing messages when there's a threshold set for the server
-              if queued_message.message.inspected == 0 && queued_message.server.outbound_spam_threshold
-                log "#{log_prefix} Inspecting message"
-                queued_message.message.inspect_message
-                if queued_message.message.inspected == 1
-                  if queued_message.message.spam_score >= queued_message.server.outbound_spam_threshold
-                    queued_message.message.update(:spam => 1)
-                  end
-                  log "#{log_prefix} Message inspected successfully"
-                end
-              end
-
-              if queued_message.message.spam == 1
-                queued_message.message.create_delivery("HardFail", :details => "Message is likely spam. Threshold is #{queued_message.server.outbound_spam_threshold} and the message scored #{queued_message.message.spam_score}.")
-                queued_message.destroy
-                log "#{log_prefix} Message is spam (#{queued_message.message.spam_score}). Hard failing."
-                next
-              end
-
-              # Add outgoing headers
-              if !queued_message.message.has_outgoing_headers?
-                queued_message.message.add_outgoing_headers
-              end
-
-              # Check send limits
-              if queued_message.server.send_limit_exceeded?
-                # If we're over the limit, we're going to be holding this message
-                queued_message.server.update_columns(:send_limit_exceeded_at => Time.now, :send_limit_approaching_at => nil)
-                queued_message.message.create_delivery('Held', :details => "Message held because send limit (#{queued_message.server.send_limit}) has been reached.")
-                queued_message.destroy
-                log "#{log_prefix} Server send limit has been exceeded. Holding."
-                next
-              elsif queued_message.server.send_limit_approaching?
-                # If we're approaching the limit, just say we are but continue to process the message
-                queued_message.server.update_columns(:send_limit_approaching_at => Time.now, :send_limit_exceeded_at => nil)
-              else
-                queued_message.server.update_columns(:send_limit_approaching_at => nil, :send_limit_exceeded_at => nil)
-              end
-
-              # Update the live stats for this message.
-              queued_message.message.database.live_stats.increment(queued_message.message.scope)
-
-              # If the server is in development mode, hold it
-              if queued_message.server.mode == 'Development' && !queued_message.manual?
-                log "Server is in development mode so holding."
-                queued_message.message.create_delivery('Held', :details => "Server is in development mode.")
-                queued_message.destroy
-                log "#{log_prefix} Server is in development mode. Holding."
-                next
-              end
-
-              # Send the outgoing message to the SMTP sender
-              begin
-                if @fixed_result
-                  result = @fixed_result
-                else
-                  sender = cached_sender(Postal::SMTPSender, queued_message.message.recipient_domain, queued_message.ip_address)
-                  result = sender.send_message(queued_message.message)
-                  if result.connect_error
-                    @fixed_result = result
-                  end
-                end
-              end
-
-              #
-              # If the message has been hard failed, check to see how many other recent hard fails we've had for the address
-              # and if there are more than 2, suppress the address for 30 days.
-              #
-              if result.type == 'HardFail'
-                recent_hard_fails = queued_message.server.message_db.select(:messages, :where => {:rcpt_to => queued_message.message.rcpt_to, :status => 'HardFail', :timestamp => {:greater_than => 24.hours.ago.to_f}}, :count => true)
-                if recent_hard_fails >= 1
-                  if queued_message.server.message_db.suppression_list.add(:recipient, queued_message.message.rcpt_to, :reason => "too many hard fails")
-                    log "#{log_prefix} Added #{queued_message.message.rcpt_to} to suppression list because #{recent_hard_fails} hard fails in 24 hours"
-                    result.details += "." if result.details =~ /\.\z/
-                    result.details += " Recipient added to suppression list (too many hard fails)."
-                  end
-                end
-              end
-
-              #
-              # If a message is sent successfully, remove the users from the suppression list
-              #
-              if result.type == 'Sent'
-                if queued_message.server.message_db.suppression_list.remove(:recipient, queued_message.message.rcpt_to)
-                  log "#{log_prefix} Removed #{queued_message.message.rcpt_to} from suppression list because success"
-                  result.details += "." if result.details =~ /\.\z/
-                  result.details += " Recipient removed from suppression list."
-                end
-              end
-
-              # Log the result
-              queued_message.message.create_delivery(result.type, :details => result.details, :output => result.output, :sent_with_ssl => result.secure, :log_id => result.log_id, :time => result.time)
-              if result.retry
-                log "#{log_prefix} Message requeued for trying later."
-                queued_message.retry_later(result.retry.is_a?(Integer) ? result.retry : nil)
-              else
-                log "#{log_prefix} Processing complete"
-                queued_message.destroy
-              end
-            end
-
-          rescue => e
-            log "#{log_prefix} Internal error: #{e.class}: #{e.message}"
-            e.backtrace.each { |e| log("#{log_prefix} #{e}") }
-            queued_message.retry_later
-            log "#{log_prefix} Queued message was unlocked"
-            if defined?(Raven)
-              Raven.capture_exception(e, :extra => {:job_id => self.id, :server_id => queued_message.server_id, :message_id => queued_message.message_id})
-            end
-            if queued_message.message
-              queued_message.message.create_delivery("Error", :details => "An internal error occurred while sending this message. This message will be retried automatically. If this persists, contact support for assistance.", :output => "#{e.class}: #{e.message}", :log_id => "J-#{self.id}")
-            end
-          end
-        end
-
-      else
-        log "Couldn't get lock for message #{params['id']}. I won't do this."
-      end
-    else
-      log "No queued message with ID #{params['id']} was available for processing."
-    end
-  ensure
-    @sender&.finish rescue nil
-  end
-
-  private
-
-  def cached_sender(klass, *args)
-    @sender ||= begin
-      sender = klass.new(*args)
-      sender.start
-      sender
-    end
-  end
-end

app/jobs/webhook_delivery_job.rb

@@ -1,13 +0,0 @@
-class WebhookDeliveryJob < Postal::Job
-  def perform
-    if webhook_request = WebhookRequest.find_by_id(params['id'])
-      if webhook_request.deliver
-        log "Succesfully delivered"
-      else
-        log "Delivery failed"
-      end
-    else
-      log "No webhook request found with ID '#{params['id']}'"
-    end
-  end
-end

app/lib/dkim_header.rb

@@ -0,0 +1,130 @@
+# frozen_string_literal: true
+
+class DKIMHeader
+
+  def initialize(domain, message)
+    if domain && domain.dkim_status == "OK"
+      @domain_name = domain.name
+      @dkim_key = domain.dkim_key
+      @dkim_identifier = domain.dkim_identifier
+    else
+      @domain_name = Postal::Config.dns.return_path_domain
+      @dkim_key = Postal.signer.private_key
+      @dkim_identifier = Postal::Config.dns.dkim_identifier
+    end
+    @domain = domain
+    @message = message
+    @raw_headers, @raw_body = @message.gsub(/\r?\n/, "\r\n").split(/\r\n\r\n/, 2)
+  end
+
+  def dkim_header
+    "DKIM-Signature: v=1; " + dkim_properties.join("\r\n\t") + signature.scan(/.{1,72}/).join("\r\n\t")
+  end
+
+  private
+
+  def headers
+    @headers ||= @raw_headers.to_s.gsub(/\r?\n\s/, " ").split(/\r?\n/)
+  end
+
+  def header_names
+    normalized_headers.map { |h| h.split(":")[0].strip }
+  end
+
+  def normalized_headers
+    [].tap do |new_headers|
+      dkim_headers = headers.select do |h|
+        h.match(/
+          ^(
+            from|sender|reply-to|subject|date|message-id|to|cc|mime-version|content-type|content-transfer-encoding|
+            resent-to|resent-cc|resent-from|resent-sender|resent-message-id|in-reply-to|references|list-id|list-help|
+            list-owner|list-unsubscribe|list-unsubscribe-post|list-subscribe|list-post
+          ):/ix)
+      end
+      dkim_headers.each do |h|
+        new_headers << normalize_header(h)
+      end
+    end
+  end
+
+  def normalize_header(content)
+    content = content.dup
+
+    # From the DKIM RFC6376
+    # https://datatracker.ietf.org/doc/html/rfc6376#section-3.4.2
+
+    # Split the key and value.
+    key, value = content.split(":", 2)
+
+    # Convert all header field names (not the header field values) to
+    # lowercase.  For example, convert "SUBJect: AbC" to "subject: AbC".
+    key.downcase!
+
+    # Unfold all header field continuation lines as described in [RFC5322]
+    value.gsub!(/\r?\n[ \t]+/, " ")
+
+    # Convert all sequences of one or more WSP characters to a single SP character.
+    value.gsub!(/[ \t]+/, " ")
+
+    # Delete all WSP characters at the end of each unfolded header field value.
+    value.gsub!(/[ \t]*\z/, "")
+
+    # Delete any WSP characters remaining after the colon separating the header field name from the header field value.
+    value.gsub!(/\A[ \t]*/, "")
+
+    # Join together
+    key + ":" + value
+  end
+
+  def normalized_body
+    @normalized_body ||= begin
+      content = @raw_body.dup
+
+      # From the DKIM RFC6376
+      # https://datatracker.ietf.org/doc/html/rfc6376#section-3.4.4
+
+      # a. Reduce whitespace
+      #
+      # * Reduce all sequences of WSP within a line to a single SP character.
+      content.gsub!(/[ \t]+/, " ")
+
+      # * Ignore all whitespace at the end of lines.  Implementations MUST NOT
+      #   remove the CRLF at the end of the line.
+      content.gsub!(/ \r\n/, "\r\n")
+
+      # b. Ignore all empty lines at the end of the message body.
+      content.gsub!(/[ \r\n]*\z/, "")
+
+      content += "\r\n"
+      content
+    end
+  end
+
+  def body_hash
+    @body_hash ||= Base64.encode64(Digest::SHA256.digest(normalized_body)).strip
+  end
+
+  def dkim_properties
+    @dkim_properties ||= [].tap do |header|
+      header << "a=rsa-sha256; c=relaxed/relaxed;"
+      header << "d=#{@domain_name};"
+      header << "s=#{@dkim_identifier}; t=#{Time.now.utc.to_i};"
+      header << "bh=#{body_hash};"
+      header << "h=#{header_names.join(':')};"
+      header << "b="
+    end
+  end
+
+  def dkim_header_for_signing
+    "dkim-signature:v=1; #{dkim_properties.join(' ')}"
+  end
+
+  def signable_header_string
+    (normalized_headers + [dkim_header_for_signing]).join("\r\n")
+  end
+
+  def signature
+    Base64.encode64(@dkim_key.sign(OpenSSL::Digest.new("SHA256"), signable_header_string)).gsub("\n", "")
+  end
+
+end

app/lib/dns_resolver.rb

@@ -0,0 +1,163 @@
+# frozen_string_literal: true
+
+require "resolv"
+
+class DNSResolver
+
+  class LocalResolversUnavailableError < StandardError
+  end
+
+  attr_reader :nameservers
+  attr_reader :timeout
+
+  def initialize(nameservers)
+    @nameservers = nameservers
+  end
+
+  # Return all A records for the given name
+  #
+  # @param [String] name
+  # @return [Array<String>]
+  def a(name, **options)
+    get_resources(name, Resolv::DNS::Resource::IN::A, **options).map do |s|
+      s.address.to_s
+    end
+  end
+
+  # Return all AAAA records for the given name
+  #
+  # @param [String] name
+  # @return [Array<String>]
+  def aaaa(name, **options)
+    get_resources(name, Resolv::DNS::Resource::IN::AAAA, **options).map do |s|
+      s.address.to_s
+    end
+  end
+
+  # Return all TXT records for the given name
+  #
+  # @param [String] name
+  # @return [Array<String>]
+  def txt(name, **options)
+    get_resources(name, Resolv::DNS::Resource::IN::TXT, **options).map do |s|
+      s.data.to_s.strip
+    end
+  end
+
+  # Return all CNAME records for the given name
+  #
+  # @param [String] name
+  # @return [Array<String>]
+  def cname(name, **options)
+    get_resources(name, Resolv::DNS::Resource::IN::CNAME, **options).map do |s|
+      s.name.to_s.downcase
+    end
+  end
+
+  # Return all MX records for the given name
+  #
+  # @param [String] name
+  # @return [Array<Array<Integer, String>>]
+  def mx(name, **options)
+    records = get_resources(name, Resolv::DNS::Resource::IN::MX, **options).map do |m|
+      [m.preference.to_i, m.exchange.to_s]
+    end
+    records.sort do |a, b|
+      if a[0] == b[0]
+        [-1, 1].sample
+      else
+        a[0] <=> b[0]
+      end
+    end
+  end
+
+  # Return the effective nameserver names for a given domain name.
+  #
+  # @param [String] name
+  # @return [Array<String>]
+  def effective_ns(name, **options)
+    records = []
+    parts = name.split(".")
+    (parts.size - 1).times do |n|
+      d = parts[n, parts.size - n + 1].join(".")
+
+      records = get_resources(d, Resolv::DNS::Resource::IN::NS, **options).map do |s|
+        s.name.to_s
+      end
+
+      break if records.present?
+    end
+
+    records
+  end
+
+  # Return the hostname for a given IP address.
+  # Returns the IP address itself if no hostname can be determined.
+  #
+  # @param [String] ip_address
+  # @return [String]
+  def ip_to_hostname(ip_address, **options)
+    dns(**options) do |dns|
+      dns.getname(ip_address)&.to_s
+    end
+  rescue Resolv::ResolvError => e
+    raise if e.message =~ /timeout/ && options[:raise_timeout_errors]
+
+    ip_address
+  end
+
+  private
+
+  def dns(raise_timeout_errors: false)
+    Resolv::DNS.open(nameserver: @nameservers,
+                     raise_timeout_errors: raise_timeout_errors) do |dns|
+      dns.timeouts = [
+        Postal::Config.dns.timeout,
+        Postal::Config.dns.timeout / 2,
+        Postal::Config.dns.timeout / 2,
+      ]
+      yield dns
+    end
+  end
+
+  def get_resources(name, type, **options)
+    encoded_name = DomainName::Punycode.encode_hostname(name)
+    dns(**options) do |dns|
+      dns.getresources(encoded_name, type)
+    end
+  end
+
+  class << self
+
+    # Return a resolver which will use the nameservers for the given domain
+    #
+    # @param [String] name
+    # @return [DNSResolver]
+    def for_domain(name)
+      nameservers = local.effective_ns(name)
+      ips = nameservers.map do |ns|
+        local.a(ns)
+      end.flatten.uniq
+      new(ips)
+    end
+
+    # Return a local resolver to use for lookups
+    #
+    # @return [DNSResolver]
+    def local
+      @local ||= begin
+        resolv_conf_path = Postal::Config.dns.resolv_conf_path
+        raise LocalResolversUnavailableError, "No resolver config found at #{resolv_conf_path}" unless File.file?(resolv_conf_path)
+
+        resolv_conf = Resolv::DNS::Config.parse_resolv_conf(resolv_conf_path)
+        if resolv_conf.nil? || resolv_conf[:nameserver].nil? || resolv_conf[:nameserver].empty?
+          raise LocalResolversUnavailableError, "Could not find nameservers in #{resolv_conf_path}"
+        end
+
+        new(resolv_conf[:nameserver])
+      end
+    end
+
+  end
+
+end

app/lib/message_dequeuer.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+module MessageDequeuer
+
+  class << self
+
+    def process(message, logger:)
+      processor = InitialProcessor.new(message, logger: logger)
+      processor.process
+    end
+
+  end
+
+end

app/lib/message_dequeuer/base.rb

@@ -0,0 +1,108 @@
+# frozen_string_literal: true
+
+module MessageDequeuer
+  class Base
+
+    class StopProcessing < StandardError
+    end
+
+    attr_reader :queued_message
+    attr_reader :logger
+    attr_reader :state
+
+    def initialize(queued_message, logger:, state: nil)
+      @queued_message = queued_message
+      @logger = logger
+      @state = state || State.new
+    end
+
+    def process
+      raise NotImplemented
+    end
+
+    class << self
+
+      def process(message, **kwargs)
+        new(message, **kwargs).process
+      end
+
+    end
+
+    private
+
+    def stop_processing
+      raise StopProcessing
+    end
+
+    def catch_stops
+      yield if block_given?
+      true
+    rescue StopProcessing
+      false
+    end
+
+    def remove_from_queue
+      @queued_message.destroy
+    end
+
+    def create_delivery(type, **kwargs)
+      @queued_message.message.create_delivery(type, **kwargs)
+    end
+
+    def log(text, **tags)
+      logger.info text, **tags
+    end
+
+    def increment_live_stats
+      queued_message.message.database.live_stats.increment(queued_message.message.scope)
+    end
+
+    def hold_if_server_development_mode
+      return if queued_message.manual?
+      return unless queued_message.server.mode == "Development"
+
+      log "server is in development mode, holding"
+      create_delivery "Held", details: "Server is in development mode."
+      remove_from_queue
+      stop_processing
+    end
+
+    def log_sender_result
+      log_details = @result.details
+
+      if @additional_delivery_details
+        log_details += "." unless log_details =~ /\.\z/
+        log_details += " "
+        log_details += @additional_delivery_details
+      end
+
+      create_delivery @result.type, details: log_details,
+                                    output: @result.output&.strip,
+                                    sent_with_ssl: @result.secure,
+                                    log_id: @result.log_id,
+                                    time: @result.time
+    end
+
+    def handle_exception(exception)
+      log "internal error: #{exception.class}: #{exception.message}"
+      exception.backtrace.each { |line| log(line) }
+
+      queued_message.retry_later unless queued_message.destroyed?
+      log "message requeued for trying later, at #{queued_message.retry_after}"
+
+      if defined?(Sentry)
+        Sentry.capture_exception(exception, extra: {
+          server_id: queued_message.server_id,
+          queued_message_id: queued_message.message_id
+        })
+      end
+
+      queued_message.message&.create_delivery("Error",
+                                              details: "An internal error occurred while sending " \
+                                                       "this message. This message will be retried " \
+                                                       "automatically.",
+                                              output: "#{exception.class}: #{exception.message}")
+    end
+
+  end
+end

app/lib/message_dequeuer/incoming_message_processor.rb

@@ -0,0 +1,215 @@
+# frozen_string_literal: true
+
+module MessageDequeuer
+  class IncomingMessageProcessor < Base
+
+    attr_reader :route
+
+    def process
+      log "message is incoming"
+
+      catch_stops do
+        handle_bounces
+        increment_live_stats
+        inspect_message
+        fail_if_spam
+        hold_if_server_development_mode
+        find_route
+        hold_or_reject_spam
+        accept_mail_without_endpoints
+        hold_messages
+        bounce_messages
+        send_message_to_sender
+        send_bounce_on_hard_fail
+        log_sender_result
+        finish_processing
+      end
+    rescue StandardError => e
+      handle_exception(e)
+    end
+
+    private
+
+    def handle_bounces
+      return unless queued_message.message.bounce
+
+      log "message is a bounce"
+      original_messages = queued_message.message.original_messages
+      unless original_messages.empty?
+        queued_message.message.original_messages.each do |orig_msg|
+          queued_message.message.update(bounce_for_id: orig_msg.id, domain_id: orig_msg.domain_id)
+          create_delivery "Processed", details: "This has been detected as a bounce message for <msg:#{orig_msg.id}>."
+          orig_msg.bounce!(queued_message.message)
+          log "bounce linked with message #{orig_msg.id}"
+        end
+        remove_from_queue
+        stop_processing
+      end
+
+      # This message was sent to the return path but hasn't been matched
+      # to an original message. If we have a route for this, route it
+      # otherwise we'll drop at this point.
+      return unless queued_message.message.route_id.nil?
+
+      log "no source messages found, hard failing"
+      create_delivery "HardFail", details: "This message was a bounce but we couldn't link it with any outgoing message and there was no route for it."
+      remove_from_queue
+      stop_processing
+    end
+
+    def inspect_message
+      return if queued_message.message.inspected
+
+      log "inspecting message"
+      queued_message.message.inspect_message
+      return unless queued_message.message.inspected
+
+      is_spam = queued_message.message.spam_score > queued_message.server.spam_threshold
+      if is_spam
+        queued_message.message.update(spam: true)
+        log "message is spam (scored #{queued_message.message.spam_score}, threshold is #{queued_message.server.spam_threshold})"
+      end
+
+      queued_message.message.append_headers(
+        "X-Postal-Spam: #{queued_message.message.spam ? 'yes' : 'no'}",
+        "X-Postal-Spam-Threshold: #{queued_message.server.spam_threshold}",
+        "X-Postal-Spam-Score: #{queued_message.message.spam_score}",
+        "X-Postal-Threat: #{queued_message.message.threat ? 'yes' : 'no'}"
+      )
+      log "message inspected, headers added", spam: queued_message.message.spam?, spam_score: queued_message.message.spam_score, threat: queued_message.message.threat?
+    end
+
+    def fail_if_spam
+      return if queued_message.message.spam_score < queued_message.server.spam_failure_threshold
+
+      log "message has a spam score higher than the server's maxmimum, hard failing", server_threshold: queued_message.server.spam_failure_threshold
+      create_delivery "HardFail",
+                      details: "Message's spam score is higher than the failure threshold for this server. " \
+                               "Threshold is currently #{queued_message.server.spam_failure_threshold}."
+      remove_from_queue
+      stop_processing
+    end
+
+    def find_route
+      @route = queued_message.message.route
+      return if @route
+
+      log "no route and/or endpoint available for processing, hard failing"
+      create_delivery "HardFail", details: "Message does not have a route and/or endpoint available for delivery."
+      remove_from_queue
+      stop_processing
+    end
+
+    def hold_or_reject_spam
+      return unless queued_message.message.spam
+      return if queued_message.manual?
+
+      case @route.spam_mode
+      when "Quarantine"
+        log "message is spam and route says to quarantine spam message, holding"
+        create_delivery "Held", details: "Message placed into quarantine."
+      when "Fail"
+        log "message is spam and route says to fail spam message, hard failing"
+        create_delivery "HardFail", details: "Message is spam and the route specifies it should be failed."
+      else
+        return
+      end
+
+      remove_from_queue
+      stop_processing
+    end
+
+    def accept_mail_without_endpoints
+      return unless @route.mode == "Accept"
+
+      log "route says to accept without endpoint, marking as processed"
+      create_delivery "Processed", details: "Message has been accepted but not sent to any endpoints."
+      remove_from_queue
+      stop_processing
+    end
+
+    def hold_messages
+      return unless @route.mode == "Hold"
+
+      if queued_message.manual?
+        log "route says to hold and message was queued manually, marking as processed"
+        create_delivery "Processed", details: "Message has been processed."
+      else
+        log "route says to hold, marking as held"
+        create_delivery "Held", details: "Message has been accepted but not sent to any endpoints."
+      end
+
+      remove_from_queue
+      stop_processing
+    end
+
+    def bounce_messages
+      return unless route.mode == "Bounce" || route.mode == "Reject"
+
+      log "route says to bounce, hard failing and sending bounce"
+
+      if id = queued_message.send_bounce
+        log "bounce sent with id #{id}"
+        create_delivery "HardFail", details: "Message has been bounced because the route asks for this. See message <msg:#{id}>"
+      end
+
+      remove_from_queue
+      stop_processing
+    end
+
+    def send_message_to_sender
+      @result = @state.send_result
+      return if @result
+
+      case queued_message.message.endpoint
+      when SMTPEndpoint
+        sender = @state.sender_for(SMTPSender, queued_message.message.recipient_domain, nil, servers: [queued_message.message.endpoint.to_smtp_client_server])
+      when HTTPEndpoint
+        sender = @state.sender_for(HTTPSender, queued_message.message.endpoint)
+      when AddressEndpoint
+        sender = @state.sender_for(SMTPSender, queued_message.message.endpoint.domain, nil, rcpt_to: queued_message.message.endpoint.address)
+      else
+        log "invalid endpoint for route (#{queued_message.message.endpoint_type})"
+        create_delivery "HardFail", details: "Invalid endpoint for route."
+        remove_from_queue
+        stop_processing
+      end
+
+      @result = sender.send_message(queued_message.message)
+      return unless @result.connect_error
+
+      @state.send_result = @result
+    end
+
+    def send_bounce_on_hard_fail
+      return unless @result.type == "HardFail"
+
+      if @result.suppress_bounce
+        log "suppressing bounce message after hard fail"
+        return
+      end
+
+      return unless queued_message.message.send_bounces?
+
+      log "sending a bounce because message hard failed"
+      return unless bounce_id = queued_message.send_bounce
+
+      @additional_delivery_details = "Sent bounce message to sender (see message <msg:#{bounce_id}>)"
+    end
+
+    def finish_processing
+      if @result.retry
+        queued_message.retry_later(@result.retry.is_a?(Integer) ? @result.retry : nil)
+        log "message requeued for trying later, at #{queued_message.retry_after}"
+        queued_message.allocate_ip_address
+        queued_message.update_column(:ip_address_id, queued_message.ip_address&.id)
+        stop_processing
+      end
+
+      log "message processing completed"
+      queued_message.message.endpoint.mark_as_used
+      remove_from_queue
+    end
+
+  end
+end

app/lib/message_dequeuer/initial_processor.rb

@@ -0,0 +1,74 @@
+# frozen_string_literal: true
+
+module MessageDequeuer
+  class InitialProcessor < Base
+
+    include HasPrometheusMetrics
+
+    attr_accessor :send_result
+
+    def process
+      logger.tagged(original_queued_message: @queued_message.id) do
+        logger.info "starting message unqueue"
+        begin
+          catch_stops do
+            increment_dequeue_metric
+            check_message_exists
+            check_message_is_ready
+            find_other_messages_for_batch
+
+            # Process the original message and then all of those
+            # found for batching.
+            process_message(@queued_message)
+            @other_messages&.each { |message| process_message(message) }
+          end
+        ensure
+          @state.finished
+        end
+        logger.info "finished message unqueue"
+      end
+    end
+
+    private
+
+    def increment_dequeue_metric
+      time_in_queue = Time.now.to_f - @queued_message.created_at.to_f
+      log "queue latency is #{time_in_queue}s"
+      observe_prometheus_histogram :postal_message_queue_latency,
+                                   time_in_queue
+    end
+
+    def check_message_exists
+      return if @queued_message.message
+
+      log "unqueue because backend message has been removed."
+      remove_from_queue
+      stop_processing
+    end
+
+    def check_message_is_ready
+      return if @queued_message.ready?
+
+      log "skipping because message isn't ready for processing"
+      @queued_message.unlock
+      stop_processing
+    end
+
+    def find_other_messages_for_batch
+      return unless Postal::Config.postal.batch_queued_messages?
+
+      @other_messages = @queued_message.batchable_messages(100)
+      log "found #{@other_messages.size} associated messages to process at the same time", batch_key: @queued_message.batch_key
+    rescue StandardError
+      @queued_message.unlock
+      raise
+    end
+
+    def process_message(queued_message)
+      logger.tagged(queued_message: queued_message.id) do
+        SingleMessageProcessor.process(queued_message, logger: @logger, state: @state)
+      end
+    end
+
+  end
+end

app/lib/message_dequeuer/outgoing_message_processor.rb

@@ -0,0 +1,190 @@
+# frozen_string_literal: true
+
+module MessageDequeuer
+  class OutgoingMessageProcessor < Base
+
+    def process
+      catch_stops do
+        check_domain
+        check_rcpt_to
+        add_tag
+        hold_if_credential_is_set_to_hold
+        hold_if_recipient_on_suppression_list
+        parse_content
+        inspect_message
+        fail_if_spam
+        add_outgoing_headers
+        check_send_limits
+        increment_live_stats
+        hold_if_server_development_mode
+        send_message_to_sender
+        add_recipient_to_suppression_list_on_too_many_hard_fails
+        remove_recipient_from_suppression_list_on_success
+        log_sender_result
+        finish_processing
+      end
+    rescue StandardError => e
+      handle_exception(e)
+    end
+
+    private
+
+    def check_domain
+      return if queued_message.message.domain
+
+      log "message has no domain, hard failing"
+      create_delivery "HardFail", details: "Message's domain no longer exist"
+      remove_from_queue
+      stop_processing
+    end
+
+    def check_rcpt_to
+      return unless queued_message.message.rcpt_to.blank?
+
+      log "message has no 'to' address, hard failing"
+      create_delivery "HardFail", details: "Message doesn't have an RCPT to"
+      remove_from_queue
+      stop_processing
+    end
+
+    def add_tag
+      return if queued_message.message.tag
+      return unless tag = queued_message.message.headers["x-postal-tag"]
+
+      log "added tag: #{tag.last}"
+      queued_message.message.update(tag: tag.last)
+    end
+
+    def hold_if_credential_is_set_to_hold
+      return if queued_message.manual?
+      return if queued_message.message.credential.nil?
+      return unless queued_message.message.credential.hold?
+
+      log "credential wants us to hold messages, holding"
+      create_delivery "Held", details: "Credential is configured to hold all messages authenticated by it."
+      remove_from_queue
+      stop_processing
+    end
+
+    def hold_if_recipient_on_suppression_list
+      return if queued_message.manual?
+      return unless sl = queued_message.server.message_db.suppression_list.get(:recipient, queued_message.message.rcpt_to)
+
+      log "recipient is on the suppression list, holding"
+      create_delivery "Held", details: "Recipient (#{queued_message.message.rcpt_to}) is on the suppression list (reason: #{sl['reason']})"
+      remove_from_queue
+      stop_processing
+    end
+
+    def parse_content
+      return unless queued_message.message.should_parse?
+
+      log "parsing message content as it hasn't been parsed before"
+      queued_message.message.parse_content
+    end
+
+    def inspect_message
+      return if queued_message.message.inspected
+      return unless queued_message.server.outbound_spam_threshold
+
+      log "inspecting message"
+      queued_message.message.inspect_message
+      return unless queued_message.message.inspected
+
+      if queued_message.message.spam_score >= queued_message.server.outbound_spam_threshold
+        queued_message.message.update(spam: true)
+      end
+
+      log "message inspected successfully", spam: queued_message.message.spam?, spam_score: queued_message.message.spam_score
+    end
+
+    def fail_if_spam
+      return unless queued_message.message.spam
+
+      log "message is spam (#{queued_message.message.spam_score}), hard failing", server_threshold: queued_message.server.outbound_spam_threshold
+      create_delivery "HardFail",
+                      details: "Message is likely spam. Threshold is #{queued_message.server.outbound_spam_threshold} and " \
+                               "the message scored #{queued_message.message.spam_score}."
+      remove_from_queue
+      stop_processing
+    end
+
+    def add_outgoing_headers
+      return if queued_message.message.has_outgoing_headers?
+
+      queued_message.message.add_outgoing_headers
+    end
+
+    def check_send_limits
+      if queued_message.server.send_limit_exceeded?
+        # If we're over the limit, we're going to be holding this message
+        log "server send limit has been exceeded, holding", send_limit: queued_message.server.send_limit
+        queued_message.server.update_columns(send_limit_exceeded_at: Time.now, send_limit_approaching_at: nil)
+        create_delivery "Held", details: "Message held because send limit (#{queued_message.server.send_limit}) has been reached."
+        remove_from_queue
+        stop_processing
+      elsif queued_message.server.send_limit_approaching?
+        # If we're approaching the limit, just say we are but continue to process the message
+        queued_message.server.update_columns(send_limit_approaching_at: Time.now, send_limit_exceeded_at: nil)
+      else
+        queued_message.server.update_columns(send_limit_approaching_at: nil, send_limit_exceeded_at: nil)
+      end
+    end
+
+    def send_message_to_sender
+      @result = @state.send_result
+      return if @result
+
+      sender = @state.sender_for(SMTPSender,
+                                 queued_message.message.recipient_domain,
+                                 queued_message.ip_address)
+
+      @result = sender.send_message(queued_message.message)
+      return unless @result.connect_error
+
+      @state.send_result = @result
+    end
+
+    def add_recipient_to_suppression_list_on_too_many_hard_fails
+      return unless @result.type == "HardFail"
+
+      recent_hard_fails = queued_message.server.message_db.select(:messages,
+                                                                  where: {
+                                                                    rcpt_to: queued_message.message.rcpt_to,
+                                                                    status: "HardFail",
+                                                                    timestamp: { greater_than: 24.hours.ago.to_f }
+                                                                  },
+                                                                  count: true)
+      return if recent_hard_fails < 1
+
+      added = queued_message.server.message_db.suppression_list.add(:recipient, queued_message.message.rcpt_to,
+                                                                    reason: "too many hard fails")
+      return unless added
+
+      log "Added #{queued_message.message.rcpt_to} to suppression list because #{recent_hard_fails} hard fails in 24 hours"
+      @additional_delivery_details = "Recipient added to suppression list (too many hard fails)"
+    end
+
+    def remove_recipient_from_suppression_list_on_success
+      return unless @result.type == "Sent"
+
+      removed = queued_message.server.message_db.suppression_list.remove(:recipient, queued_message.message.rcpt_to)
+      return unless removed
+
+      log "removed #{queued_message.message.rcpt_to} from suppression list"
+      @additional_delivery_details = "Recipient removed from suppression list"
+    end
+
+    def finish_processing
+      if @result.retry
+        queued_message.retry_later(@result.retry.is_a?(Integer) ? @result.retry : nil)
+        log "message requeued for trying later", retry_after: queued_message.retry_after
+        stop_processing
+      end
+
+      log "message processing complete"
+      remove_from_queue
+    end
+
+  end
+end

app/lib/message_dequeuer/single_message_processor.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+module MessageDequeuer
+  class SingleMessageProcessor < Base
+
+    def process
+      catch_stops do
+        check_message_exists
+        check_server_suspension
+        check_delivery_attempts
+        check_raw_message_exists
+
+        processor = nil
+        case queued_message.message.scope
+        when "incoming"
+          processor = IncomingMessageProcessor
+        when "outgoing"
+          processor = OutgoingMessageProcessor
+        else
+          create_delivery "HardFail", details: "Scope #{queued_message.message.scope} is not valid"
+          remove_from_queue
+          stop_processing
+        end
+
+        processor.process(queued_message, logger: @logger, state: @state)
+      end
+    rescue StandardError => e
+      handle_exception(e)
+    end
+
+    private
+
+    def check_message_exists
+      return if queued_message.message
+
+      log "unqueueing because backend message has been removed"
+      remove_from_queue
+      stop_processing
+    end
+
+    def check_server_suspension
+      return unless queued_message.server.suspended?
+
+      log "server is suspended, holding message"
+      create_delivery "Held", details: "Mail server has been suspended. No e-mails can be processed at present. Contact support for assistance."
+      remove_from_queue
+      stop_processing
+    end
+
+    def check_delivery_attempts
+      return if queued_message.attempts < Postal::Config.postal.default_maximum_delivery_attempts
+
+      details = "Maximum number of delivery attempts (#{queued_message.attempts}) has been reached."
+      if queued_message.message.scope == "incoming"
+        # Send bounces to incoming e-mails when they are hard failed
+        if bounce_id = queued_message.send_bounce
+          details += " Bounce sent to sender (see message <msg:#{bounce_id}>)"
+        end
+      elsif queued_message.message.scope == "outgoing"
+        # Add the recipient to the suppression list
+        if queued_message.server.message_db.suppression_list.add(:recipient, queued_message.message.rcpt_to, reason: "too many soft fails")
+          log "added #{queued_message.message.rcpt_to} to suppression list because maximum attempts has been reached"
+          details += " Added #{queued_message.message.rcpt_to} to suppression list because delivery has failed #{queued_message.attempts} times."
+        end
+      end
+
+      log "message has reached maximum number of attempts, hard failing"
+      create_delivery "HardFail", details: details
+      remove_from_queue
+      stop_processing
+    end
+
+    def check_raw_message_exists
+      return if queued_message.message.raw_message?
+
+      log "raw message has been removed, not sending"
+      create_delivery "HardFail", details: "Raw message has been removed. Cannot send message."
+      remove_from_queue
+      stop_processing
+    end
+
+  end
+end

app/lib/message_dequeuer/state.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module MessageDequeuer
+  class State
+
+    attr_accessor :send_result
+
+    def sender_for(klass, *args, **kwargs)
+      @cached_senders ||= {}
+      @cached_senders[[klass, args, kwargs]] ||= begin
+        klass_instance = klass.new(*args, **kwargs)
+        klass_instance.start
+        klass_instance
+      end
+    end
+
+    def finished
+      @cached_senders&.each_value do |sender|
+        sender.finish
+      rescue StandardError
+        false
+      end
+    end
+
+  end
+end

app/lib/query_string.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+class QueryString
+
+  def initialize(string)
+    @string = string.strip + " "
+  end
+
+  def [](value)
+    hash[value.to_s]
+  end
+
+  delegate :empty?, to: :hash
+
+  def hash
+    @hash ||= @string.scan(/([a-z]+):\s*(?:(\d{2,4}-\d{2}-\d{2}\s\d{2}:\d{2})|"(.*?)"|(.*?))(\s|\z)/).each_with_object({}) do |(key, date, string_with_spaces, value), hash|
+      if date
+        actual_value = date
+      elsif string_with_spaces
+        actual_value = string_with_spaces
+      elsif value == "[blank]"
+        actual_value = nil
+      else
+        actual_value = value
+      end
+
+      if hash.keys.include?(key.to_s)
+        hash[key.to_s] = [hash[key.to_s]].flatten
+        hash[key.to_s] << actual_value
+      else
+        hash[key.to_s] = actual_value
+      end
+    end
+  end
+
+end

app/lib/received_header.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+class ReceivedHeader
+
+  OUR_HOSTNAMES = {
+    smtp: Postal::Config.postal.smtp_hostname,
+    http: Postal::Config.postal.web_hostname
+  }.freeze
+
+  class << self
+
+    def generate(server, helo, ip_address, method)
+      our_hostname = OUR_HOSTNAMES[method]
+      if our_hostname.nil?
+        raise Error, "`method` is invalid (must be one of #{OUR_HOSTNAMES.join(', ')})"
+      end
+
+      header = "by #{our_hostname} with #{method.to_s.upcase}; #{Time.now.utc.rfc2822}"
+
+      if server.nil? || server.privacy_mode == false
+        hostname = DNSResolver.local.ip_to_hostname(ip_address)
+        header = "from #{helo} (#{hostname} [#{ip_address}]) #{header}"
+      end
+
+      header
+    end
+
+  end
+
+end

app/lib/reply_separator.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+class ReplySeparator
+
+  RULES = [
+    /^-{2,10} $.*/m,
+    /^>*\s*----- ?Original Message ?-----.*/m,
+    /^>*\s*From:[^\r\n]*[\r\n]+Sent:.*/m,
+    /^>*\s*From:[^\r\n]*[\r\n]+Date:.*/m,
+    /^>*\s*-----Urspr.ngliche Nachricht----- .*/m,
+    /^>*\s*Le[^\r\n]{10,200}a .crit ?:\s*$.*/,
+    /^>*\s*__________________.*/m,
+    /^>*\s*On.{10,200}wrote:\s*$.*/m,
+    /^>*\s*Sent from my.*/m,
+    /^>*\s*=== Please reply above this line ===.*/m,
+    /(^>.*\n?){10,}/,
+  ].freeze
+
+  def self.separate(text)
+    return "" unless text.is_a?(String)
+
+    text = text.gsub("\r", "")
+    stripped = String.new
+    RULES.each do |rule|
+      text.gsub!(rule) do
+        stripped = ::Regexp.last_match(0).to_s + "\n" + stripped
+        ""
+      end
+    end
+    stripped = stripped.strip
+    [text.strip, stripped.presence]
+  end
+
+end

app/lib/smtp_client/endpoint.rb

@@ -0,0 +1,169 @@
+# frozen_string_literal: true
+
+module SMTPClient
+  class Endpoint
+
+    class SMTPSessionNotStartedError < StandardError
+    end
+
+    attr_reader :server
+    attr_reader :ip_address
+    attr_accessor :smtp_client
+
+    # @param server [Server] the server that this IP address is for
+    # @param ip_address [String] the IP address
+    def initialize(server, ip_address)
+      @server = server
+      @ip_address = ip_address
+    end
+
+    # Return a description of this server with its IP address
+    #
+    # @return [String]
+    def description
+      "#{@ip_address}:#{@server.port} (#{@server.hostname})"
+    end
+
+    # Return a string representation of this server
+    #
+    # @return [String]
+    def to_s
+      description
+    end
+
+    # Return true if this is an IPv6 address
+    #
+    # @return [Boolean]
+    def ipv6?
+      @ip_address.include?(":")
+    end
+
+    # Return true if this is an IPv4 address
+    #
+    # @return [Boolean]
+    def ipv4?
+      !ipv6?
+    end
+
+    # Start a new SMTP session and store the client with this server for future use as needed
+    #
+    # @param source_ip_address [IPAddress] the IP address to use as the source address for the connection
+    # @param allow_ssl [Boolean] whether to allow SSL for this connection, if false SSL mode is ignored
+    #
+    # @return [Net::SMTP]
+    def start_smtp_session(source_ip_address: nil, allow_ssl: true)
+      @smtp_client = Net::SMTP.new(@ip_address, @server.port)
+      @smtp_client.open_timeout = Postal::Config.smtp_client.open_timeout
+      @smtp_client.read_timeout = Postal::Config.smtp_client.read_timeout
+      @smtp_client.tls_hostname = @server.hostname
+
+      if source_ip_address
+        @source_ip_address = source_ip_address
+      end
+
+      if @source_ip_address
+        @smtp_client.source_address = ipv6? ? @source_ip_address.ipv6 : @source_ip_address.ipv4
+      end
+
+      if allow_ssl
+        case @server.ssl_mode
+        when SSLModes::AUTO
+          @smtp_client.enable_starttls_auto(self.class.ssl_context_without_verify)
+        when SSLModes::STARTTLS
+          @smtp_client.enable_starttls(self.class.ssl_context_with_verify)
+        when SSLModes::TLS
+          @smtp_client.enable_tls(self.class.ssl_context_with_verify)
+        else
+          @smtp_client.disable_starttls
+          @smtp_client.disable_tls
+        end
+      else
+        @smtp_client.disable_starttls
+        @smtp_client.disable_tls
+      end
+
+      @smtp_client.start(@source_ip_address ? @source_ip_address.hostname : self.class.default_helo_hostname)
+
+      @smtp_client
+    end
+
+    # Send a message to the current SMTP session (or create one if there isn't one for this endpoint).
+    # If sending messsage encouters some connection errors, retry again after re-establishing the SMTP
+    # session.
+    #
+    # @param raw_message [String] the raw message to send
+    # @param mail_from [String] the MAIL FROM address
+    # @param rcpt_to [String] the RCPT TO address
+    # @param retry_on_connection_error [Boolean] whether to retry the connection if there is a connection error
+    #
+    # @return [void]
+    def send_message(raw_message, mail_from, rcpt_to, retry_on_connection_error: true)
+      raise SMTPSessionNotStartedError if @smtp_client.nil? || (@smtp_client && !@smtp_client.started?)
+
+      @smtp_client.rset_errors
+      @smtp_client.send_message(raw_message, mail_from, [rcpt_to])
+    rescue Errno::ECONNRESET, Errno::EPIPE, OpenSSL::SSL::SSLError
+      if retry_on_connection_error
+        finish_smtp_session
+        start_smtp_session
+        return send_message(raw_message, mail_from, rcpt_to, retry_on_connection_error: false)
+      end
+
+      raise
+    end
+
+    # Reset the current SMTP session for this server if possible otherwise
+    # finish the session
+    #
+    # @return [void]
+    def reset_smtp_session
+      @smtp_client&.rset
+    rescue StandardError
+      finish_smtp_session
+    end
+
+    # Finish the current SMTP session for this server if possible.
+    #
+    # @return [void]
+    def finish_smtp_session
+      @smtp_client&.finish
+    rescue StandardError
+      nil
+    ensure
+      @smtp_client = nil
+    end
+
+    class << self
+
+      # Return the default HELO hostname to present to SMTP servers that
+      # we connect to
+      #
+      # @return [String]
+      def default_helo_hostname
+        Postal::Config.dns.helo_hostname ||
+          Postal::Config.postal.smtp_hostname ||
+          "localhost"
+      end
+
+      def ssl_context_with_verify
+        @ssl_context_with_verify ||= begin
+          c = OpenSSL::SSL::SSLContext.new
+          c.verify_mode = OpenSSL::SSL::VERIFY_PEER
+          c.cert_store = OpenSSL::X509::Store.new
+          c.cert_store.set_default_paths
+          c
+        end
+      end
+
+      def ssl_context_without_verify
+        @ssl_context_without_verify ||= begin
+          c = OpenSSL::SSL::SSLContext.new
+          c.verify_mode = OpenSSL::SSL::VERIFY_NONE
+          c
+        end
+      end
+
+    end
+
+  end
+end

app/lib/smtp_client/server.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+module SMTPClient
+  class Server
+
+    attr_reader :hostname
+    attr_reader :port
+    attr_accessor :ssl_mode
+
+    def initialize(hostname, port: 25, ssl_mode: SSLModes::AUTO)
+      @hostname = hostname
+      @port = port
+      @ssl_mode = ssl_mode
+    end
+
+    # Return all IP addresses for this server by resolving its hostname.
+    # IPv6 addresses will be returned first.
+    #
+    # @return [Array<SMTPClient::Endpoint>]
+    def endpoints
+      ips = []
+
+      DNSResolver.local.aaaa(@hostname).each do |ip|
+        ips << Endpoint.new(self, ip)
+      end
+
+      DNSResolver.local.a(@hostname).each do |ip|
+        ips << Endpoint.new(self, ip)
+      end
+
+      ips
+    end
+
+  end
+end

app/lib/smtp_client/ssl_modes.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+module SMTPClient
+  module SSLModes
+
+    AUTO = "Auto"
+    STARTTLS = "STARTLS"
+    TLS = "TLS"
+    NONE = "None"
+
+  end
+end

app/lib/smtp_server/client.rb

@@ -0,0 +1,597 @@
+# frozen_string_literal: true
+
+module SMTPServer
+  class Client
+
+    extend HasPrometheusMetrics
+    include HasPrometheusMetrics
+
+    CRAM_MD5_DIGEST = OpenSSL::Digest.new("md5")
+    LOG_REDACTION_STRING = "[redacted]"
+
+    attr_reader :logging_enabled
+    attr_reader :credential
+    attr_reader :ip_address
+    attr_reader :recipients
+    attr_reader :headers
+    attr_reader :state
+    attr_reader :helo_name
+
+    def initialize(ip_address)
+      @logging_enabled = true
+      @ip_address = ip_address
+
+      @cr_present = false
+      @previous_cr_present = nil
+
+      if @ip_address
+        check_ip_address
+        @state = :welcome
+      else
+        @state = :preauth
+      end
+      transaction_reset
+    end
+
+    def check_ip_address
+      return unless @ip_address &&
+                    Postal::Config.smtp_server.log_ip_address_exclusion_matcher &&
+                    @ip_address =~ Regexp.new(Postal::Config.smtp_server.log_ip_address_exclusion_matcher)
+
+      @logging_enabled = false
+    end
+
+    def transaction_reset
+      @recipients = []
+      @mail_from = nil
+      @data = nil
+      @headers = nil
+    end
+
+    def trace_id
+      @trace_id ||= SecureRandom.alphanumeric(8).upcase
+    end
+
+    def handle(data)
+      if data[-1] == "\r"
+        @cr_present = true
+        data = data.chop # remove last character (\r)
+      else
+        # This doesn't use `logger` because that will be nil when logging is disabled
+        # and we always want to log this.
+        Postal.logger&.warn("Detected line with invalid line ending (missing <CR>)", trace_id: trace_id)
+        @cr_present = false
+      end
+
+      if @state == :preauth
+        return proxy(data)
+      end
+
+      logger&.debug "\e[32m<= #{sanitize_input_for_log(data.strip)}\e[0m"
+      if @proc
+        @proc.call(data)
+      else
+        handle_command(data)
+      end
+    ensure
+      @previous_cr_present = @cr_present
+    end
+
+    def finished?
+      @finished || false
+    end
+
+    def start_tls?
+      @start_tls || false
+    end
+
+    attr_writer :start_tls
+
+    def handle_command(data)
+      case data
+      when /^QUIT/i           then quit
+      when /^STARTTLS/i       then starttls
+      when /^EHLO/i           then ehlo(data)
+      when /^HELO/i           then helo(data)
+      when /^RSET/i           then rset
+      when /^NOOP/i           then noop
+      when /^AUTH PLAIN/i     then auth_plain(data)
+      when /^AUTH LOGIN/i     then auth_login(data)
+      when /^AUTH CRAM-MD5/i  then auth_cram_md5(data)
+      when /^MAIL FROM/i      then mail_from(data)
+      when /^RCPT TO/i        then rcpt_to(data)
+      when /^DATA/i           then data(data)
+      else
+        increment_error_count("invalid-command")
+        "502 Invalid/unsupported command"
+      end
+    end
+
+    def logger
+      return nil unless @logging_enabled
+
+      @logger ||= Postal.logger.create_tagged_logger(trace_id: trace_id)
+    end
+
+    private
+
+    def proxy(data)
+      # inet-protocol, client-ip, proxy-ip, client-port, proxy-port
+      if m = data.match(/\APROXY (.+) (.+) (.+) (.+) (.+)\z/)
+        @ip_address = m[2]
+        check_ip_address
+        @state = :welcome
+        logger&.debug "\e[35mClient identified as #{@ip_address}\e[0m"
+        increment_command_count("PROXY")
+        return "220 #{Postal::Config.postal.smtp_hostname} ESMTP Postal/#{trace_id}"
+      end
+
+      @finished = true
+      increment_error_count("proxy-error")
+      "502 Proxy Error"
+    end
+
+    def quit
+      @finished = true
+      "221 Closing Connection"
+    end
+
+    def starttls
+      if Postal::Config.smtp_server.tls_enabled?
+        @start_tls = true
+        @tls = true
+        increment_command_count("STARTLS")
+        "220 Ready to start TLS"
+      else
+        increment_error_count("tls-unavailable")
+        "502 TLS not available"
+      end
+    end
+
+    def ehlo(data)
+      @helo_name = data.strip.split(" ", 2)[1]
+      transaction_reset
+      @state = :welcomed
+      increment_command_count("EHLO")
+      [
+        "250-My capabilities are",
+        Postal::Config.smtp_server.tls_enabled? && !@tls ? "250-STARTTLS" : nil,
+        "250 AUTH CRAM-MD5 PLAIN LOGIN",
+      ].compact
+    end
+
+    def helo(data)
+      @helo_name = data.strip.split(" ", 2)[1]
+      transaction_reset
+      @state = :welcomed
+      increment_command_count("HELO")
+      "250 #{Postal::Config.postal.smtp_hostname}"
+    end
+
+    def rset
+      transaction_reset
+      @state = :welcomed
+      increment_command_count("RSET")
+      "250 OK"
+    end
+
+    def noop
+      "250 OK"
+    end
+
+    def auth_plain(data)
+      increment_command_count("AUTH PLAIN")
+
+      handler = proc do |idata|
+        @proc = nil
+        idata = Base64.decode64(idata)
+        parts = idata.split("\0")
+        username = parts[-2]
+        password = parts[-1]
+        unless username && password
+          increment_error_count("missing-credentials")
+          next "535 Authenticated failed - protocol error"
+        end
+
+        authenticate(password)
+      end
+
+      data = data.gsub(/AUTH PLAIN ?/i, "")
+      if data.strip == ""
+        @proc = handler
+        @password_expected_next = true
+        "334"
+      else
+        handler.call(data)
+      end
+    end
+
+    def auth_login(data)
+      increment_command_count("AUTH LOGIN")
+
+      password_handler = proc do |idata|
+        @proc = nil
+        password = Base64.decode64(idata)
+        authenticate(password)
+      end
+
+      username_handler = proc do
+        @proc = password_handler
+        @password_expected_next = true
+        "334 UGFzc3dvcmQ6" # "Password:"
+      end
+
+      data = data.gsub(/AUTH LOGIN ?/i, "")
+      if data.strip == ""
+        @proc = username_handler
+        "334 VXNlcm5hbWU6" # "Username:"
+      else
+        username_handler.call(nil)
+      end
+    end
+
+    def authenticate(password)
+      if @credential = Credential.where(type: "SMTP", key: password).first
+        @credential.use
+        "235 Granted for #{@credential.server.organization.permalink}/#{@credential.server.permalink}"
+      else
+        logger&.warn "Authentication failure for #{@ip_address}"
+        increment_error_count("invalid-credentials")
+        "535 Invalid credential"
+      end
+    end
+
+    def auth_cram_md5(data)
+      increment_command_count("AUTH CRAM-MD5")
+
+      challenge = Digest::SHA1.hexdigest(Time.now.to_i.to_s + rand(100_000).to_s)
+      challenge = "<#{challenge[0, 20]}@#{Postal::Config.postal.smtp_hostname}>"
+
+      handler = proc do |idata|
+        @proc = nil
+        username, password = Base64.decode64(idata).split(" ", 2).map { |a| a.chomp }
+        org_permlink, server_permalink = username.split(/[\/_]/, 2)
+        server = ::Server.includes(:organization).where(organizations: { permalink: org_permlink }, permalink: server_permalink).first
+        if server.nil?
+          logger&.warn "Authentication failure for #{@ip_address} (no server found matching #{username})"
+          increment_error_count("invalid-credentials")
+          next "535 Denied"
+        end
+
+        grant = nil
+        server.credentials.where(type: "SMTP").each do |credential|
+          correct_response = OpenSSL::HMAC.hexdigest(CRAM_MD5_DIGEST, credential.key, challenge)
+          next unless password == correct_response
+
+          @credential = credential
+          @credential.use
+          logger&.debug "Authenticated with with credential #{credential.id}"
+          grant = "235 Granted for #{credential.server.organization.permalink}/#{credential.server.permalink}"
+          break
+        end
+
+        if grant.nil?
+          logger&.warn "Authentication failure for #{@ip_address} (invalid credential)"
+          increment_error_count("invalid-credentials")
+          next "535 Denied"
+        end
+
+        grant
+      end
+
+      @proc = handler
+      "334 " + Base64.encode64(challenge).gsub(/[\r\n]/, "")
+    end
+
+    def mail_from(data)
+      unless in_state(:welcomed, :mail_from_received)
+        increment_error_count("mail-from-out-of-order")
+        return "503 EHLO/HELO first please"
+      end
+
+      @state = :mail_from_received
+      transaction_reset
+      if data =~ /AUTH=/
+        # Discard AUTH= parameter and anything that follows.
+        # We don't need this parameter as we don't trust any client to set it
+        mail_from_line = data.sub(/ *AUTH=.*/, "")
+      else
+        mail_from_line = data
+      end
+      @mail_from = mail_from_line.gsub(/MAIL FROM\s*:\s*/i, "").gsub(/.*</, "").gsub(/>.*/, "").strip
+      "250 OK"
+    end
+
+    def rcpt_to(data)
+      unless in_state(:mail_from_received, :rcpt_to_received)
+        increment_error_count("rcpt-to-out-of-order")
+        return "503 EHLO/HELO and MAIL FROM first please"
+      end
+
+      rcpt_to = data.gsub(/RCPT TO\s*:\s*/i, "").gsub(/.*</, "").gsub(/>.*/, "").strip
+
+      if rcpt_to.blank?
+        increment_error_count("empty-rcpt-to")
+        return "501 RCPT TO should not be empty"
+      end
+
+      uname, domain = rcpt_to.split("@", 2)
+
+      if domain.blank?
+        increment_error_count("invalid-rcpt-to")
+        return "501 Invalid RCPT TO"
+      end
+
+      uname, tag = uname.split("+", 2)
+
+      if domain == Postal::Config.dns.return_path_domain || domain =~ /\A#{Regexp.escape(Postal::Config.dns.custom_return_path_prefix)}\./
+        # This is a return path
+        @state = :rcpt_to_received
+        if server = ::Server.where(token: uname).first
+          if server.suspended?
+            increment_error_count("server-suspended")
+            "535 Mail server has been suspended"
+          else
+            logger&.debug "Added bounce on server #{server.id}"
+            @recipients << [:bounce, rcpt_to, server]
+            "250 OK"
+          end
+        else
+          increment_error_count("invalid-server-token")
+          "550 Invalid server token"
+        end
+
+      elsif domain == Postal::Config.dns.route_domain
+        # This is an email direct to a route. This isn't actually supported yet.
+        @state = :rcpt_to_received
+        if route = Route.where(token: uname).first
+          if route.server.suspended?
+            increment_error_count("server-suspended")
+            "535 Mail server has been suspended"
+          elsif route.mode == "Reject"
+            increment_error_count("route-rejected")
+            "550 Route does not accept incoming messages"
+          else
+            logger&.debug "Added route #{route.id} to recipients (tag: #{tag.inspect})"
+            actual_rcpt_to = "#{route.name}#{tag ? "+#{tag}" : ''}@#{route.domain.name}"
+            @recipients << [:route, actual_rcpt_to, route.server, { route: route }]
+            "250 OK"
+          end
+        else
+          "550 Invalid route token"
+        end
+
+      elsif @credential
+        # This is outgoing mail for an authenticated user
+        @state = :rcpt_to_received
+        if @credential.server.suspended?
+          increment_error_count("server-suspended")
+          "535 Mail server has been suspended"
+        else
+          logger&.debug "Added external address '#{rcpt_to}'"
+          @recipients << [:credential, rcpt_to, @credential.server]
+          "250 OK"
+        end
+
+      elsif uname && domain && route = Route.find_by_name_and_domain(uname, domain)
+        # This is incoming mail for a route
+        @state = :rcpt_to_received
+        if route.server.suspended?
+          increment_error_count("server-suspended")
+          "535 Mail server has been suspended"
+        elsif route.mode == "Reject"
+          increment_error_count("route-rejection")
+          "550 Route does not accept incoming messages"
+        else
+          logger&.debug "Added route #{route.id} to recipients (tag: #{tag.inspect})"
+          @recipients << [:route, rcpt_to, route.server, { route: route }]
+          "250 OK"
+        end
+
+      else
+        # User is trying to relay but is not authenticated. Try to authenticate by IP address
+        @credential = Credential.where(type: "SMTP-IP").all.sort_by { |c| c.ipaddr&.prefix || 0 }.reverse.find do |credential|
+          credential.ipaddr.include?(@ip_address) || (credential.ipaddr.ipv4? && credential.ipaddr.ipv4_mapped.include?(@ip_address))
+        end
+
+        if @credential
+          # Retry with credential
+          @credential.use
+          rcpt_to(data)
+        else
+          increment_error_count("authentication-required")
+          logger&.warn "Authentication failure for #{@ip_address}"
+          "530 Authentication required"
+        end
+      end
+    end
+
+    def data(_data)
+      unless in_state(:rcpt_to_received)
+        increment_error_count("data-out-of-order")
+        return "503 HELO/EHLO, MAIL FROM and RCPT TO before sending data"
+      end
+
+      @data = String.new.force_encoding("BINARY")
+      @headers = {}
+      @receiving_headers = true
+
+      received_header = ReceivedHeader.generate(@credential&.server, @helo_name, @ip_address, :smtp)
+                                      .force_encoding("BINARY")
+
+      @data << "Received: #{received_header}\r\n"
+      @headers["received"] = [received_header]
+
+      handler = proc do |idata|
+        if idata == "." && @cr_present && @previous_cr_present
+          @logging_enabled = true
+          @proc = nil
+          finished
+        else
+          idata = idata.to_s.sub(/\A\.\./, ".")
+
+          if @credential&.server&.log_smtp_data?
+            # We want to log if enabled
+          else
+            logger&.debug "Not logging further message data."
+            @logging_enabled = false
+          end
+
+          if @receiving_headers
+            if idata&.length&.zero?
+              @receiving_headers = false
+            elsif idata.to_s =~ /^\s/
+              # This is a continuation of a header
+              if @header_key && @headers[@header_key.downcase] && @headers[@header_key.downcase].last
+                @headers[@header_key.downcase].last << idata.to_s
+              end
+            else
+              @header_key, value = idata.split(/:\s*/, 2)
+              @headers[@header_key.downcase] ||= []
+              @headers[@header_key.downcase] << value
+            end
+          end
+          @data << idata
+          @data << "\r\n"
+          nil
+        end
+      end
+
+      @proc = handler
+      "354 Go ahead"
+    end
+
+    def finished
+      if @data.bytesize > Postal::Config.smtp_server.max_message_size.megabytes.to_i
+        transaction_reset
+        @state = :welcomed
+        increment_error_count("message-too-large")
+        return format("552 Message too large (maximum size %dMB)", Postal::Config.smtp_server.max_message_size)
+      end
+
+      if @headers["received"].grep(/by #{Postal::Config.postal.smtp_hostname}/).count > 4
+        transaction_reset
+        @state = :welcomed
+        increment_error_count("loop-detected")
+        return "550 Loop detected"
+      end
+
+      authenticated_domain = nil
+      if @credential
+        authenticated_domain = @credential.server.find_authenticated_domain_from_headers(@headers)
+        if authenticated_domain.nil?
+          transaction_reset
+          @state = :welcomed
+          increment_error_count("from-name-invalid")
+          return "530 From/Sender name is not valid"
+        end
+      end
+
+      @recipients.each do |recipient|
+        type, rcpt_to, server, options = recipient
+
+        case type
+        when :credential
+          increment_message_count("outgoing")
+
+          # Outgoing messages are just inserted
+          message = server.message_db.new_message
+          message.rcpt_to = rcpt_to
+          message.mail_from = @mail_from
+          message.raw_message = @data
+          message.received_with_ssl = @tls
+          message.scope = "outgoing"
+          message.domain_id = authenticated_domain&.id
+          message.credential_id = @credential.id
+          message.save
+
+        when :bounce
+          increment_message_count("bounce")
+          if rp_route = server.routes.where(name: "__returnpath__").first
+            # If there's a return path route, we can use this to create the message
+            rp_route.create_messages do |msg|
+              msg.rcpt_to = rcpt_to
+              msg.mail_from = @mail_from
+              msg.raw_message = @data
+              msg.received_with_ssl = @tls
+              msg.bounce = 1
+            end
+          else
+            # There's no return path route, we just need to insert the mesage
+            # without going through the route.
+            message = server.message_db.new_message
+            message.rcpt_to = rcpt_to
+            message.mail_from = @mail_from
+            message.raw_message = @data
+            message.received_with_ssl = @tls
+            message.scope = "incoming"
+            message.bounce = 1
+            message.save
+          end
+        when :route
+          increment_message_count("incoming")
+          options[:route].create_messages do |msg|
+            msg.rcpt_to = rcpt_to
+            msg.mail_from = @mail_from
+            msg.raw_message = @data
+            msg.received_with_ssl = @tls
+          end
+        end
+      end
+      transaction_reset
+      @state = :welcomed
+      "250 OK"
+    end
+
+    def in_state(*states)
+      states.include?(@state)
+    end
+
+    def sanitize_input_for_log(data)
+      if @password_expected_next
+        @password_expected_next = false
+        if data =~ /\A[a-z0-9]{3,}=*\z/i
+          return LOG_REDACTION_STRING
+        end
+      end
+
+      data = data.dup
+      data.gsub!(/(.*AUTH \w+) (.*)\z/i) { "#{::Regexp.last_match(1)} #{LOG_REDACTION_STRING}" }
+      data
+    end
+
+    def increment_error_count(error)
+      increment_prometheus_counter :postal_smtp_server_client_errors, labels: { error: error }
+    end
+
+    def increment_command_count(command)
+      increment_prometheus_counter :postal_smtp_server_commands_total, labels: { command: command }
+    end
+
+    def increment_message_count(type)
+      increment_prometheus_counter :postal_smtp_server_messages_total, labels: {
+        type: type,
+        tls: @tls ? "yes" : "no"
+      }
+    end
+
+    class << self
+
+      def register_prometheus_metrics
+        register_prometheus_counter :postal_smtp_server_commands_total,
+                                    docstring: "The number of key commands received by the server",
+                                    labels: [:command]
+
+        register_prometheus_counter :postal_smtp_server_client_errors,
+                                    docstring: "The number of errors sent to a client",
+                                    labels: [:error]
+
+        register_prometheus_counter :postal_smtp_server_messages_total,
+                                    docstring: "The number of messages accepted by the SMTP server",
+                                    labels: [:type, :tls]
+      end
+
+    end
+
+  end
+end

app/lib/smtp_server/server.rb

@@ -0,0 +1,317 @@
+# frozen_string_literal: true
+
+require "ipaddr"
+require "nio"
+
+module SMTPServer
+  class Server
+
+    include HasPrometheusMetrics
+
+    class << self
+
+      def tls_private_key
+        @tls_private_key ||= OpenSSL::PKey.read(File.read(Postal::Config.smtp_server.tls_private_key_path))
+      end
+
+      def tls_certificates
+        @tls_certificates ||= begin
+          data = File.read(Postal::Config.smtp_server.tls_certificate_path)
+          certs = data.scan(/-----BEGIN CERTIFICATE-----.+?-----END CERTIFICATE-----/m)
+          certs.map do |c|
+            OpenSSL::X509::Certificate.new(c)
+          end.freeze
+        end
+      end
+
+    end
+
+    def initialize(options = {})
+      @options = options
+      @options[:debug] ||= false
+      register_prometheus_metrics
+      prepare_environment
+    end
+
+    def run
+      logger.tagged(component: "smtp-server") do
+        listen
+        run_event_loop
+      end
+    end
+
+    private
+
+    def prepare_environment
+      $\ = "\r\n"
+      BasicSocket.do_not_reverse_lookup = true
+
+      trap("TERM") do
+        $stdout.puts "Received TERM signal, shutting down."
+        unlisten
+      end
+
+      trap("INT") do
+        $stdout.puts "Received INT signal, shutting down."
+        unlisten
+      end
+    end
+
+    def ssl_context
+      @ssl_context ||= begin
+        ssl_context      = OpenSSL::SSL::SSLContext.new
+        ssl_context.cert = self.class.tls_certificates[0]
+        ssl_context.extra_chain_cert = self.class.tls_certificates[1..]
+        ssl_context.key = self.class.tls_private_key
+        ssl_context.ssl_version = Postal::Config.smtp_server.ssl_version if Postal::Config.smtp_server.ssl_version
+        ssl_context.ciphers = Postal::Config.smtp_server.tls_ciphers if Postal::Config.smtp_server.tls_ciphers
+        ssl_context
+      end
+    end
+
+    def listen
+      bind_address = ENV.fetch("BIND_ADDRESS", Postal::Config.smtp_server.default_bind_address)
+      port = ENV.fetch("PORT", Postal::Config.smtp_server.default_port)
+
+      @server = TCPServer.open(bind_address, port)
+      @server.autoclose = false
+      @server.close_on_exec = false
+      if defined?(Socket::SOL_SOCKET) && defined?(Socket::SO_KEEPALIVE)
+        @server.setsockopt(Socket::SOL_SOCKET, Socket::SO_KEEPALIVE, true)
+      end
+      if defined?(Socket::SOL_TCP) && defined?(Socket::TCP_KEEPIDLE) && defined?(Socket::TCP_KEEPINTVL) && defined?(Socket::TCP_KEEPCNT)
+        @server.setsockopt(Socket::SOL_TCP, Socket::TCP_KEEPIDLE, 50)
+        @server.setsockopt(Socket::SOL_TCP, Socket::TCP_KEEPINTVL, 10)
+        @server.setsockopt(Socket::SOL_TCP, Socket::TCP_KEEPCNT, 5)
+      end
+
+      logger.info "Listening on #{bind_address}:#{port}"
+    end
+
+    def unlisten
+      # Instruct the nio loop to unlisten and wake it
+      @unlisten = true
+      @io_selector.wakeup
+    end
+
+    def run_event_loop
+      # Set up an instance of nio4r to monitor for connections and data
+      @io_selector = NIO::Selector.new
+      # Register the SMTP listener
+      @io_selector.register(@server, :r)
+      # Create a hash to contain a buffer for each client.
+      buffers = Hash.new { |h, k| h[k] = String.new.force_encoding("BINARY") }
+      loop do
+        # Wait for an event to occur
+        @io_selector.select do |monitor|
+          # Get the IO from the nio monitor
+          io = monitor.io
+          # Is this event an incoming connection?
+          if io.is_a?(TCPServer)
+            begin
+              # Accept the connection
+              new_io = io.accept
+              increment_prometheus_counter :postal_smtp_server_connections_total
+              # Get the client's IP address and strip `::ffff:` for consistency.
+              client_ip_address = new_io.remote_address.ip_address.sub(/\A::ffff:/, "")
+              if Postal::Config.smtp_server.proxy_protocol?
+                # If we are using the haproxy proxy protocol, we will be sent the
+                # client's IP later. Delay the welcome process.
+                client = Client.new(nil)
+                if Postal::Config.smtp_server.log_connections?
+                  client.logger&.debug "Connection opened from #{client_ip_address}"
+                end
+              else
+                # We're not using the proxy protocol so we already know the client's IP
+                client = Client.new(client_ip_address)
+                if Postal::Config.smtp_server.log_connections?
+                  client.logger&.debug "Connection opened from #{client_ip_address}"
+                end
+                # We know who the client is, welcome them.
+                client.logger&.debug "Client identified as #{client_ip_address}"
+                new_io.print("220 #{Postal::Config.postal.smtp_hostname} ESMTP Postal/#{client.trace_id}")
+              end
+              # Register the client and its socket with nio4r
+              monitor = @io_selector.register(new_io, :r)
+              monitor.value = client
+            rescue StandardError => e
+              # If something goes wrong, log as appropriate and disconnect the client
+              if defined?(Sentry)
+                Sentry.capture_exception(e, extra: { trace_id: begin
+                  client.trace_id
+                rescue StandardError
+                  nil
+                end })
+              end
+              logger.error "An error occurred while accepting a new client."
+              logger.error "#{e.class}: #{e.message}"
+              e.backtrace.each do |line|
+                logger.error line
+              end
+              increment_prometheus_counter :postal_smtp_server_exceptions_total,
+                                           labels: { error: e.class.to_s, type: "client-accept" }
+              begin
+                new_io.close
+              rescue StandardError
+                nil
+              end
+            end
+          else
+            # This event is not an incoming connection so it must be data from a client
+            begin
+              # Get the client from the nio monitor
+              client = monitor.value
+              # For now we assume the connection isn't closed
+              eof = false
+              # Is the client negotiating a TLS handshake?
+              if client.start_tls?
+                begin
+                  # Can we accept the TLS connection at this time?
+                  io.accept_nonblock
+                  # Increment prometheus
+                  increment_prometheus_counter :postal_smtp_server_tls_connections_total
+                  # We were able to accept the connection, the client is no longer handshaking
+                  client.start_tls = false
+                rescue IO::WaitReadable, IO::WaitWritable => e
+                  # Could not accept without blocking
+                  # We will try again later
+                  next
+                rescue OpenSSL::SSL::SSLError => e
+                  client.logger&.debug "SSL Negotiation Failed: #{e.message}"
+                  eof = true
+                end
+              else
+                # The client is not negotiating a TLS handshake at this time
+                begin
+                  # Read 10kiB of data at a time from the socket.
+                  buffers[io] << io.readpartial(10_240)
+
+                  # There is an extra step for SSL sockets
+                  if io.is_a?(OpenSSL::SSL::SSLSocket)
+                    buffers[io] << io.readpartial(10_240) while io.pending.positive?
+                  end
+                rescue EOFError, Errno::ECONNRESET, Errno::ETIMEDOUT
+                  # Client went away
+                  eof = true
+                end
+
+                # We line buffer, so look to see if we have received a newline
+                # and keep doing so until all buffered lines have been processed.
+                while buffers[io].index("\n")
+                  # Extract the line
+                  line, buffers[io] = buffers[io].split("\n", 2)
+                  # Send the received line to the client object for processing
+                  result = client.handle(line)
+                  # If the client object returned some data, write it back to the client
+                  next if result.nil?
+
+                  result = [result] unless result.is_a?(Array)
+                  result.compact.each do |iline|
+                    client.logger&.debug "\e[34m=> #{iline.strip}\e[0m"
+                    begin
+                      io.write(iline.to_s + "\r\n")
+                      io.flush
+                    rescue Errno::ECONNRESET
+                      # Client disconnected before we could write response
+                      eof = true
+                    end
+                  end
+                end
+
+                # Did the client request STARTTLS?
+                if !eof && client.start_tls?
+                  # Deregister the unencrypted IO
+                  @io_selector.deregister(io)
+                  buffers.delete(io)
+                  io = OpenSSL::SSL::SSLSocket.new(io, ssl_context)
+                  # Close the underlying IO when the TLS socket is closed
+                  io.sync_close = true
+                  # Register the new TLS socket with nio
+                  monitor = @io_selector.register(io, :r)
+                  monitor.value = client
+                end
+              end
+
+              # Has the client requested we close the connection?
+              if client.finished? || eof
+                client.logger&.debug "Connection closed"
+                # Deregister the socket and close it
+                @io_selector.deregister(io)
+                buffers.delete(io)
+                io.close
+                # If we have no more clients or listeners left, exit the process
+                if @io_selector.empty?
+                  Process.exit(0)
+                end
+              end
+            rescue StandardError => e
+              # Something went wrong, log as appropriate
+              client_id = client ? client.trace_id : "------"
+              if defined?(Sentry)
+                Sentry.capture_exception(e, extra: { trace_id: begin
+                  client&.trace_id
+                rescue StandardError
+                  nil
+                end })
+              end
+              logger.error "An error occurred while processing data from a client.", trace_id: client_id
+              logger.error "#{e.class}: #{e.message}", trace_id: client_id
+              e.backtrace.each do |iline|
+                logger.error iline, trace_id: client_id
+              end
+
+              increment_prometheus_counter :postal_smtp_server_exceptions_total,
+                                           labels: { error: e.class.to_s, type: "data" }
+
+              # Close all IO and forget this client
+              begin
+                @io_selector.deregister(io)
+              rescue StandardError
+                nil
+              end
+              buffers.delete(io)
+              begin
+                io.close
+              rescue StandardError
+                nil
+              end
+              if @io_selector.empty?
+                Process.exit(0)
+              end
+            end
+          end
+        end
+        # If unlisten has been called, stop listening
+        next unless @unlisten
+
+        @io_selector.deregister(@server)
+        @server.close
+        # If there's nothing left to do, shut down the process
+        if @io_selector.empty?
+          Process.exit(0)
+        end
+        # Clear the request
+        @unlisten = false
+      end
+    end
+
+    def logger
+      Postal.logger
+    end
+
+    def register_prometheus_metrics
+      register_prometheus_counter :postal_smtp_server_connections_total,
+                                  docstring: "The number of connections made to the Postal SMTP server."
+
+      register_prometheus_counter :postal_smtp_server_exceptions_total,
+                                  docstring: "The number of server exceptions encountered by the SMTP server",
+                                  labels: [:type, :error]
+
+      register_prometheus_counter :postal_smtp_server_tls_connections_total,
+                                  docstring: "The number of successfuly TLS connections established"
+
+      Client.register_prometheus_metrics
+    end
+
+  end
+end

app/lib/worker/jobs/base_job.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module Worker
+  module Jobs
+    class BaseJob
+
+      def initialize(logger:)
+        @logger = logger
+      end
+
+      def call
+        # Override me.
+      end
+
+      def work_completed?
+        @work_completed == true
+      end
+
+      private
+
+      def work_completed!
+        @work_completed = true
+      end
+
+      attr_reader :logger
+
+    end
+  end
+end

app/lib/worker/jobs/process_queued_messages_job.rb

@@ -0,0 +1,73 @@
+# frozen_string_literal: true
+
+module Worker
+  module Jobs
+    class ProcessQueuedMessagesJob < BaseJob
+
+      def call
+        @lock_time = Time.current
+        @locker = Postal.locker_name_with_suffix(SecureRandom.hex(8))
+
+        find_ip_addresses
+        lock_message_for_processing
+        obtain_locked_messages
+        process_messages
+        @messages_to_process
+      end
+
+      private
+
+      # Returns an array of IP address IDs that are present on the host that is
+      # running this job.
+      #
+      # @return [Array<Integer>]
+      def find_ip_addresses
+        ip_addresses = { 4 => [], 6 => [] }
+        Socket.ip_address_list.each do |address|
+          next if local_ip?(address.ip_address)
+
+          ip_addresses[address.ipv4? ? 4 : 6] << address.ip_address
+        end
+        @ip_addresses = IPAddress.where(ipv4: ip_addresses[4]).or(IPAddress.where(ipv6: ip_addresses[6])).pluck(:id)
+      end
+
+      # Is the given IP address a local address?
+      #
+      # @param [String] ip
+      # @return [Boolean]
+      def local_ip?(ip)
+        !!(ip =~ /\A(127\.|fe80:|::)/)
+      end
+
+      # Obtain a queued message from the database for processing
+      #
+      # @return [void]
+      def lock_message_for_processing
+        QueuedMessage.where(ip_address_id: [nil, @ip_addresses])
+                     .where(locked_by: nil, locked_at: nil)
+                     .ready_with_delayed_retry
+                     .limit(1)
+                     .update_all(locked_by: @locker, locked_at: @lock_time)
+      end
+
+      # Get a full list of all messages which we can process (i.e. those which have just
+      # been locked by us for processing)
+      #
+      # @return [void]
+      def obtain_locked_messages
+        @messages_to_process = QueuedMessage.where(locked_by: @locker, locked_at: @lock_time)
+      end
+
+      # Process the messages we obtained from the database
+      #
+      # @return [void]
+      def process_messages
+        @messages_to_process.each do |message|
+          work_completed!
+          MessageDequeuer.process(message, logger: logger)
+        end
+      end
+
+    end
+  end
+end

app/lib/worker/jobs/process_webhook_requests_job.rb

@@ -0,0 +1,49 @@
+# frozen_string_literal: true
+
+module Worker
+  module Jobs
+    class ProcessWebhookRequestsJob < BaseJob
+
+      def call
+        @lock_time = Time.current
+        @locker = Postal.locker_name_with_suffix(SecureRandom.hex(8))
+
+        lock_request_for_processing
+        obtain_locked_requests
+        process_requests
+      end
+
+      private
+
+      # Obtain a webhook request from the database for processing
+      #
+      # @return [void]
+      def lock_request_for_processing
+        WebhookRequest.unlocked
+                      .ready
+                      .limit(1)
+                      .update_all(locked_by: @locker, locked_at: @lock_time)
+      end
+
+      # Get a full list of all webhooks which we can process (i.e. those which have just
+      # been locked by us for processing)
+      #
+      # @return [void]
+      def obtain_locked_requests
+        @requests_to_process = WebhookRequest.where(locked_by: @locker, locked_at: @lock_time)
+      end
+
+      # Process the webhook requests we obtained from the database
+      #
+      # @return [void]
+      def process_requests
+        @requests_to_process.each do |request|
+          work_completed!
+
+          WebhookDeliveryService.new(webhook_request: request).call
+        end
+      end
+
+    end
+  end
+end

app/lib/worker/process.rb

@@ -0,0 +1,317 @@
+# frozen_string_literal: true
+
+module Worker
+  # The Postal Worker process is responsible for handling all background tasks. This includes processing of all
+  # messages, webhooks and other administrative tasks. There are two main types of background work which is completed,
+  # jobs and scheduled tasks.
+  #
+  # The 'Jobs' here allow for the continuous monitoring of a database table (or queue) and processing of any new items
+  # which may appear in that. The polling takes place every 5 seconds by default and the work is able to run multiple
+  # threads to look for and process this work.
+  #
+  # Scheduled Tasks allow for code to be executed on a ROUGH schedule. This is used for administrative tasks.  A single
+  # thread will run within each worker process and attempt to acquire the 'tasks' role. If successful it will run all
+  # tasks which are due to be run. The tasks are then scheduled to run again at a future time. Workers which are not
+  # successful in acquiring the role will not run any tasks but will still attempt to acquire a lock in case the current
+  # acquiree disappears.
+  #
+  # The worker process will run until it receives a TERM or INT signal. It will then attempt to gracefully shut down
+  # after it has completed any outstanding jobs which are already inflight.
+  class Process
+
+    include HasPrometheusMetrics
+
+    # An array of job classes that should be processed each time the worker ticks.
+    #
+    # @return [Array<Class>]
+    JOBS = [
+      Jobs::ProcessQueuedMessagesJob,
+      Jobs::ProcessWebhookRequestsJob,
+    ].freeze
+
+    # An array of tasks that should be processed
+    #
+    # @return [Array<Class>]
+    TASKS = [
+      ActionDeletionsScheduledTask,
+      CheckAllDNSScheduledTask,
+      CleanupAuthieSessionsScheduledTask,
+      ExpireHeldMessagesScheduledTask,
+      ProcessMessageRetentionScheduledTask,
+      PruneSuppressionListsScheduledTask,
+      PruneWebhookRequestsScheduledTask,
+      SendNotificationsScheduledTask,
+      TidyQueuedMessagesTask,
+    ].freeze
+
+    # @param [Integer] thread_count The number of worker threads to run in this process
+    def initialize(thread_count: Postal::Config.worker.threads,
+                   work_sleep_time: 5,
+                   task_sleep_time: 60)
+      @thread_count = thread_count
+      @exit_pipe_read, @exit_pipe_write = IO.pipe
+      @work_sleep_time = work_sleep_time
+      @task_sleep_time = task_sleep_time
+      @threads = []
+
+      setup_prometheus
+    end
+
+    def run
+      logger.tagged(component: "worker") do
+        setup_traps
+        ensure_connection_pool_size_is_suitable
+        start_work_threads
+        start_tasks_thread
+        wait_for_threads
+      end
+    end
+
+    private
+
+    # Install signal traps to allow for graceful shutdown
+    #
+    # @return [void]
+    def setup_traps
+      trap("INT") { receive_signal("INT") }
+      trap("TERM") { receive_signal("TERM") }
+    end
+
+    # Receive a signal and set the shutdown flag
+    #
+    # @param [String] signal The signal that was received z
+    # @return [void]
+    def receive_signal(signal)
+      puts "Received #{signal} signal. Stopping when able."
+      @shutdown = true
+      @exit_pipe_write.close
+    end
+
+    # Wait for the period of time and return true or false if shutdown has been requested. If the shutdown is
+    # requested during the wait, it will return immediately otherwise it will return false when it has finished
+    # waiting for the period of time.
+    #
+    # @param [Integer] wait_time The time to wait for
+    # @return [Boolean]
+    def shutdown_after_wait?(wait_time)
+      @exit_pipe_read.wait_readable(wait_time) ? true : false
+    end
+
+    # Ensure that the connection pool is big enough for the number of threads
+    # configured.
+    #
+    # @return [void]
+    def ensure_connection_pool_size_is_suitable
+      current_pool_size = ActiveRecord::Base.connection_pool.size
+      desired_pool_size = @thread_count + 3
+
+      return if current_pool_size >= desired_pool_size
+
+      logger.warn "number of worker threads (#{@thread_count}) is more  " \
+                  "than the db connection pool size (#{current_pool_size}+3), " \
+                  "increasing connection pool size to #{desired_pool_size}"
+
+      Postal.change_database_connection_pool_size(desired_pool_size)
+    end
+
+    # Wait for all threads to complete
+    #
+    # @return [void]
+    def wait_for_threads
+      @threads.each(&:join)
+    end
+
+    # Start the worker threads
+    #
+    # @return [void]
+    def start_work_threads
+      logger.info "starting #{@thread_count} work threads"
+      @thread_count.times do |index|
+        start_work_thread(index)
+      end
+    end
+
+    # Start a worker thread
+    #
+    # @return [void]
+    def start_work_thread(index)
+      @threads << Thread.new do
+        logger.tagged(component: "worker", thread: "work#{index}") do
+          logger.info "started work thread #{index}"
+          loop do
+            work_completed = work(index)
+
+            if shutdown_after_wait?(work_completed ? 0 : @work_sleep_time)
+              break
+            end
+          end
+
+          logger.info "stopping work thread #{index}"
+        end
+      end
+    end
+
+    # Actually perform the work for this tick. This will call each job which has been registered.
+    #
+    # @return [Boolean] Whether any work was completed in this job or not
+    def work(thread)
+      completed_work = 0
+      ActiveRecord::Base.connection_pool.with_connection do
+        JOBS.each do |job_class|
+          capture_errors do
+            job = job_class.new(logger: logger)
+
+            time = Benchmark.realtime { job.call }
+
+            observe_prometheus_histogram :postal_worker_job_runtime,
+                                         time,
+                                         labels: {
+                                          thread: thread,
+                                          job: job_class.to_s.split("::").last
+                                         }
+
+            if job.work_completed?
+              completed_work += 1
+              increment_prometheus_counter :postal_worker_job_executions,
+                                           labels: {
+                                              thread: thread,
+                                              job: job_class.to_s.split("::").last
+                                           }
+            end
+          end
+        end
+      end
+      completed_work.positive?
+    end
+
+    # Start the tasks thread
+    #
+    # @return [void]
+    def start_tasks_thread
+      logger.info "starting tasks thread"
+      @threads << Thread.new do
+        logger.tagged(component: "worker", thread: "tasks") do
+          loop do
+            run_tasks
+
+            if shutdown_after_wait?(@task_sleep_time)
+              break
+            end
+          end
+
+          logger.info "stopping tasks thread"
+          ActiveRecord::Base.connection_pool.with_connection do
+            if WorkerRole.release(:tasks)
+              logger.info "released tasks role"
+            end
+          end
+        end
+      end
+    end
+
+    # Run the tasks. This will attempt to acquire the tasks role and if successful it will all the registered
+    # tasks if they are due to be run.
+    #
+    # @return [void]
+    def run_tasks
+      role_acquisition_status = ActiveRecord::Base.connection_pool.with_connection do
+        WorkerRole.acquire(:tasks)
+      end
+
+      case role_acquisition_status
+      when :stolen
+        logger.info "acquired task role by stealing it from a lazy worker"
+      when :created
+        logger.info "acquired task role by creating it"
+      when :renewed
+        logger.debug "acquired task role by renewing it"
+      else
+        logger.debug "could not acquire task role, not doing anything"
+        return false
+      end
+
+      ActiveRecord::Base.connection_pool.with_connection do
+        TASKS.each { |task| run_task(task) }
+      end
+    end
+
+    # Run a single task
+    #
+    # @param [Class] task The task to run
+    # @return [void]
+    def run_task(task)
+      logger.tagged task: task do
+        scheduled_task = ScheduledTask.find_by(name: task.to_s)
+        if scheduled_task.nil?
+          logger.info "no existing task object, creating it now"
+          scheduled_task = ScheduledTask.create!(name: task.to_s, next_run_after: task.next_run_after)
+        end
+
+        next unless scheduled_task.next_run_after < Time.current
+
+        logger.info "running task"
+
+        time = 0
+        capture_errors do
+          time = Benchmark.realtime do
+            task.new(logger: logger).call
+          end
+
+          observe_prometheus_histogram :postal_worker_task_runtime,
+                                       time,
+                                       labels: {
+                                        task: task.to_s.split("::").last
+                                       }
+        end
+
+        next_run_after = task.next_run_after
+        logger.info "scheduling task to next run at #{next_run_after}"
+        scheduled_task.update!(next_run_after: next_run_after)
+      end
+    end
+
+    # Return the logger
+    #
+    # @return [Klogger::Logger]
+    def logger
+      Postal.logger
+    end
+
+    # Capture exceptions and handle this as appropriate.
+    #
+    # @yield The block of code to run
+    # @return [void]
+    def capture_errors
+      yield
+    rescue StandardError => e
+      logger.error "#{e.class} (#{e.message})"
+      e.backtrace.each { |line| logger.error line }
+      Sentry.capture_exception(e) if defined?(Sentry)
+
+      increment_prometheus_counter :postal_worker_errors,
+                                   labels: { error: e.class.to_s }
+    end
+
+    def setup_prometheus
+      register_prometheus_counter :postal_worker_job_executions,
+                                  docstring: "The number of jobs worked by a worker where work was completed",
+                                  labels: [:thread, :job]
+
+      register_prometheus_histogram :postal_worker_job_runtime,
+                                    docstring: "The time taken to process jobs (in seconds)",
+                                    labels: [:thread, :job]
+
+      register_prometheus_counter :postal_worker_errors,
+                                  docstring: "The number of errors encountered while processing jobs",
+                                  labels: [:error]
+
+      register_prometheus_histogram :postal_worker_task_runtime,
+                                    docstring: "The time taken to process tasks (in seconds)",
+                                    labels: [:task]
+
+      register_prometheus_histogram :postal_message_queue_latency,
+                                    docstring: "The length of time between a message being queued and being dequeued (in seconds)"
+    end
+
+  end
+end

app/mailers/app_mailer.rb

@@ -1,51 +1,37 @@
+# frozen_string_literal: true
+
 class AppMailer < ApplicationMailer
 
-  def verify_email_address(user)
-    @user = user
-    mail :to => @user.email_address, :subject => "Verify your new e-mail address"
-  end
-
-  def new_user(user)
-    @user = user
-    mail :to => @user.email_address, :subject => "Welcome to Postal"
-  end
-
-  def user_invite(user_invite, organization)
-    @user_invite = user_invite
-    @organization = organization
-    mail :to => @user_invite.email_address, :subject => "Access the #{organization.name} organization on Postal"
-  end
-
   def verify_domain(domain, email_address, user)
     @domain = domain
     @email_address = email_address
     @user = user
-    mail :to => email_address, :subject => "Verify your ownership of #{@domain.name}"
+    mail to: email_address, subject: "Verify your ownership of #{@domain.name}"
   end
 
   def password_reset(user, return_to = nil)
     @user = user
     @return_to = return_to
-    mail :to => @user.email_address, :subject => "Reset your Postal password"
+    mail to: @user.email_address, subject: "Reset your Postal password"
   end
 
   def server_send_limit_approaching(server)
     @server = server
-    mail :to => @server.organization.notification_addresses, :subject => "[#{server.full_permalink}] Mail server is approaching its send limit"
+    mail to: @server.organization.notification_addresses, subject: "[#{server.full_permalink}] Mail server is approaching its send limit"
   end
 
   def server_send_limit_exceeded(server)
     @server = server
-    mail :to => @server.organization.notification_addresses, :subject => "[#{server.full_permalink}] Mail server has exceeded its send limit"
+    mail to: @server.organization.notification_addresses, subject: "[#{server.full_permalink}] Mail server has exceeded its send limit"
   end
 
   def server_suspended(server)
     @server = server
-    mail :to => @server.organization.notification_addresses, :subject => "[#{server.full_permalink}] Your mail server has been suspended"
+    mail to: @server.organization.notification_addresses, subject: "[#{server.full_permalink}] Your mail server has been suspended"
   end
 
   def test_message(recipient)
-    mail :to => recipient, :subject => "Postal SMTP Test Message"
+    mail to: recipient, subject: "Postal SMTP Test Message"
   end
 
 end

app/mailers/application_mailer.rb

@@ -1,4 +1,8 @@
+# frozen_string_literal: true
+
 class ApplicationMailer < ActionMailer::Base
-  default :from => "#{Postal.smtp_from_name} <#{Postal.smtp_from_address}>"
+
+  default from: "#{Postal::Config.smtp.from_name} <#{Postal::Config.smtp.from_address}>"
   layout false
+
 end

app/models/additional_route_endpoint.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 # == Schema Information
 #
 # Table name: additional_route_endpoints
@@ -13,22 +15,21 @@
 class AdditionalRouteEndpoint < ApplicationRecord
 
   belongs_to :route
-  belongs_to :endpoint, :polymorphic => true
+  belongs_to :endpoint, polymorphic: true
 
   validate :validate_endpoint_belongs_to_server
   validate :validate_wildcard
   validate :validate_uniqueness
 
   def self.find_by_endpoint(endpoint)
-    class_name, id = endpoint.split('#', 2)
+    class_name, id = endpoint.split("#", 2)
     unless Route::ENDPOINT_TYPES.include?(class_name)
       raise Postal::Error, "Invalid endpoint class name '#{class_name}'"
     end
-    if uuid = class_name.constantize.find_by_uuid(id)
-      where(:endpoint_type => class_name, :endpoint_id => uuid).first
-    else
-      nil
-    end
+
+    return unless uuid = class_name.constantize.find_by_uuid(id)
+
+    where(endpoint_type: class_name, endpoint_id: uuid).first
   end
 
   def _endpoint
@@ -36,41 +37,37 @@ class AdditionalRouteEndpoint < ApplicationRecord
   end
 
   def _endpoint=(value)
-    if value.blank?
-      self.endpoint = nil
-    else
-      if value =~ /\#/
-        class_name, id = value.split('#', 2)
-        unless Route::ENDPOINT_TYPES.include?(class_name)
-          raise Postal::Error, "Invalid endpoint class name '#{class_name}'"
-        end
-        self.endpoint = class_name.constantize.find_by_uuid(id)
-      else
-        self.endpoint = nil
+    if value && value =~ /\#/
+      class_name, id = value.split("#", 2)
+      unless Route::ENDPOINT_TYPES.include?(class_name)
+        raise Postal::Error, "Invalid endpoint class name '#{class_name}'"
       end
+
+      self.endpoint = class_name.constantize.find_by_uuid(id)
+    else
+      self.endpoint = nil
     end
   end
 
   private
 
   def validate_endpoint_belongs_to_server
-    if self.endpoint && self.endpoint&.server != self.route.server
-      errors.add :endpoint, :invalid
-    end
+    return unless endpoint && endpoint&.server != route.server
+
+    errors.add :endpoint, :invalid
   end
 
   def validate_uniqueness
-    if self.endpoint == self.route.endpoint
-      errors.add :base, "You can only add an endpoint to a route once"
-    end
+    return unless endpoint == route.endpoint
+
+    errors.add :base, "You can only add an endpoint to a route once"
   end
 
   def validate_wildcard
-    if self.route.wildcard?
-      if self.endpoint_type == 'SMTPEndpoint' || self.endpoint_type == 'AddressEndpoint'
-        errors.add :base, "SMTP or address endpoints are not permitted on wildcard routes"
-      end
-    end
+    return unless route.wildcard?
+    return unless endpoint_type == "SMTPEndpoint" || endpoint_type == "AddressEndpoint"
+
+    errors.add :base, "SMTP or address endpoints are not permitted on wildcard routes"
   end
 
 end

app/models/address_endpoint.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 # == Schema Information
 #
 # Table name: address_endpoints
@@ -16,10 +18,10 @@ class AddressEndpoint < ApplicationRecord
   include HasUUID
 
   belongs_to :server
-  has_many :routes, :as => :endpoint
-  has_many :additional_route_endpoints, :dependent => :destroy, :as => :endpoint
+  has_many :routes, as: :endpoint
+  has_many :additional_route_endpoints, dependent: :destroy, as: :endpoint
 
-  validates :address, :presence => true, :format => {:with => /@/}, :uniqueness => {:scope => [:server_id], :message => "has already been added"}
+  validates :address, presence: true, format: { with: /@/ }, uniqueness: { scope: [:server_id], message: "has already been added", case_sensitive: false }
 
   before_destroy :update_routes
 
@@ -28,15 +30,15 @@ class AddressEndpoint < ApplicationRecord
   end
 
   def update_routes
-    self.routes.each { |r| r.update(:endpoint => nil, :mode => 'Reject') }
+    routes.each { |r| r.update(endpoint: nil, mode: "Reject") }
   end
 
   def description
-    self.address
+    address
   end
 
   def domain
-    address.split('@', 2).last
+    address.split("@", 2).last
   end
 
 end

app/models/application_record.rb

@@ -1,5 +1,9 @@
+# frozen_string_literal: true
+
 class ApplicationRecord < ActiveRecord::Base
+
   self.abstract_class = true
-  self.inheritance_column = 'sti_type'
+  self.inheritance_column = "sti_type"
   nilify_blanks
+
 end

app/models/bounce_message.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+class BounceMessage
+
+  def initialize(server, message)
+    @server = server
+    @message = message
+  end
+
+  def raw_message
+    mail = Mail.new
+    mail.to = @message.mail_from
+    mail.from = "Mail Delivery Service <#{@message.route.description}>"
+    mail.subject = "Mail Delivery Failed (#{@message.subject})"
+    mail.text_part = body
+    mail.attachments["Original Message.eml"] = { mime_type: "message/rfc822", encoding: "quoted-printable", content: @message.raw_message }
+    mail.message_id = "<#{SecureRandom.uuid}@#{Postal::Config.dns.return_path_domain}>"
+    mail.to_s
+  end
+
+  def queue
+    message = @server.message_db.new_message
+    message.scope = "outgoing"
+    message.rcpt_to = @message.mail_from
+    message.mail_from = @message.route.description
+    message.domain_id = @message.domain&.id
+    message.raw_message = raw_message
+    message.bounce = true
+    message.bounce_for_id = @message.id
+    message.save
+    message.id
+  end
+
+  def postmaster_address
+    @server.postmaster_address || "postmaster@#{@message.domain&.name || Postal::Config.postal.web_hostname}"
+  end
+
+  private
+
+  def body
+    <<~BODY
+      This is the mail delivery service responsible for delivering mail to #{@message.route.description}.
+
+      The message you've sent cannot be delivered. Your original message is attached to this message.
+
+      For further assistance please contact #{postmaster_address}. Please include the details below to help us identify the issue.
+
+      Message Token: #{@message.token}@#{@server.token}
+      Orginal Message ID: #{@message.message_id}
+      Mail from: #{@message.mail_from}
+      Rcpt To: #{@message.rcpt_to}
+    BODY
+  end
+
+end