ActiveStorage - migrating from Carrierwave attachment

5th November 2018 – 1521 words

In a previous article we already learned how to migrate between ActiveStorage service providers. A more common usecase for many Rails app is to migrate from an existing attachment solution, like Paperclip, Carrierwave or Dragonfly to ActiveStorage.

I want to demonstrate how we did it for a recent project. We did use Carrierwave.

1. Introduce a new attachment column

We played around replacing the old attachment column (e.g. “Organisation#logo”) inline from Carrierwave to ActiveStorage, but decided against it. When having a new column, even only “logo2”, we can more easily check which calls have already been migrated. After adding migration code we can also remove the mount_uploader call and see if all tests are still passing. If something goes wrong, we can still access the old files.

class Organisation < ApplicationRecord
  mount_uploader :logo, LogoUploader
  has_one_attached :logo
end

2. Migrate the attachments

def migrate_attachment!(klass:, attachment_attribute:, carrierwave_uploader:, active_storage_column: attachment_attribute)
  klass.find_each do |item|
    next unless item.send(attachment_attribute).present?
    attachment = item.send(attachment_attribute)
    attachment.cache_stored_file!
    file = attachment.sanitized_file.file
    content_type = item.send(attachment_attribute).content_type
    item.send(active_storage_column).attach(io: file, content_type: content_type, filename: item.attributes[attachment_attribute.to_s])
    item.save
  end
end

migrate_attachment!(klass: Organisation, attachment_attribute: :logo, carrierwave_uploader: LogoUploader, active_storage_column: :logo2)

Run this script to copy all Carrierwave files to ActiveStorage. If you are using another storage solution, this part should be a little different

3. Migrate all calls to the old column

This might be tricky, the following replacement patterns have occurred so far:

Passing raw io / file objects to the object

# pattern
object.logo = File.open(...)

# replace with:
object.logo2.attach(io: File.open('...'), filename: "somefilename.jpg")

Checking if attachment exists:

# pattern
if object.logo.present?

# replace with:
if object.logo2.attached?

Getting blob/tempfile of blob

Till Rails 6, there is no released code to do that easily. Meanwhile, put this in a initializer:

# config/initializers/activestorage_patch.rb
if Rails.version > "5.2"
  raise "Check, if the ActiveStorage::Downloader is released with your current Rails version

  https://github.com/rails/rails/commit/ee21b7c2eb64def8f00887a9fafbd77b85f464f1#diff-3fd88ddd945ad24c4bd6f76c64c8790a:
  "
end
# source from https://github.com/rails/rails/blob/c823f9252be2552c65bb1370ccf42a14de461439/activestorage/lib/active_storage/downloader.rb
module ActiveStorage
  class Downloader #:nodoc:
    def initialize(blob, tempdir: nil)
      @blob    = blob
      @tempdir = tempdir
    end

    def download_blob_to_tempfile
      open_tempfile do |file|
        download_blob_to file
        verify_integrity_of file
        yield file
      end
    end

    private
      attr_reader :blob, :tempdir

      def open_tempfile
        file = Tempfile.open([ "ActiveStorage-#{blob.id}-", blob.filename.extension_with_delimiter ], tempdir)

        begin
          yield file
        ensure
          file.close!
        end
      end

      def download_blob_to(file)
        file.binmode
        blob.download { |chunk| file.write(chunk) }
        file.flush
        file.rewind
      end

      def verify_integrity_of(file)
        unless Digest::MD5.file(file).base64digest == blob.checksum
          raise ActiveStorage::IntegrityError
        end
      end
  end
end

# create a monkey-patch module to easier follow our patch
module AsDownloadPatch
  def open(tempdir: nil, &block)
    ActiveStorage::Downloader.new(self, tempdir: tempdir).download_blob_to_tempfile(&block)
  end
end

# inject patch when rails thinks it's appropriate (e.g. after Development reloading)
Rails.application.config.to_prepare do
  ActiveStorage::Blob.send(:include, AsDownloadPatch)
end

Now you can use this code like this:

if organisation.logo.attached?
  organisation.logo.open do |tempfile|
    tempfile.read
    ...
  end
end

Generating versions/variants and linking to them

# pattern
class LogoUploader < CarrierWave::Uploader::Base
  version :same_height do
    process resize_to_fill: [0, 200]
  end

# ...
image_tag object.logo.url(:medium)


# replace with:
class Organisation < ApplicationRecord # or better: some OrganisationDecorator
  def logo_same_height_variant
    if logo2.attached?
      logo2.variant(
        combine_options: {
        gravity: "center",
        resize: "1000x200>",
        crop: "1000x200+0+0"
      })
    end
  end

# ....

image_tag object.logo_same_height_variant

Linking to a variant from outside controller/view

If you need the path to a variant in a different context, e.g. Grape API, ActiveJobs etc. then you can reference the path like this:

class Organisation < ..
  def logo_same_height_path
    if logo2.attached?
      Rails.application.routes.
        url_helpers.
        rails_representation_path(logo_same_height_variant, only_path: true)
    end
  end
end

json = {
  logo: {
    same_height: organisation.logo_same_height_path
  }
}

Downloading variant blob (e.g. Base64 encoded attachments)

Getting hands on the variant blob, for Base64 encoding, custom analysis, exporting, PDF inlining etc.)

# before:
blob = organisation.logo.read
Base64.strict_encode64(blob)


# after:
# config/initializers/as_path.rb
module AsVariantDownloadPatch
  def open(tempdir: nil, &block)
    processed
    file = Tempfile.open(["ActiveStorage-#{key}-", blob.filename.extension_with_delimiter], tempdir)
    begin
      file.binmode
      service.download(key) { |chunk| file.write(chunk) }
      file.flush
      file.rewind
      yield file
    ensure
      file.close!
    end
  end
end

Rails.application.config.to_prepare do
  ActiveStorage::Variant.send(:include, AsVariantDownloadPatch)
end

organisation.logo_same_height_variant.open do |tf|
  base64 = Base64.strict_encode64(tf.read)
end

4. Deploy code to production

Deploy your changes after testing and grepping over your code base. Run your migration script from Part 2 (maybe even in a deployment before).

After some time you can delete the old attachment column and remove the mount_uploader / Uploader class.

Limitations of ActiveStorage

The following criteria must be discussed before migrating:

  • Until Rails 6, AS has no built-in validation. So all validations logic must be implement by oneself
  • No support for image optimization (image_optim, pngquant, jpegoptim) at the moment.
  • You need ActiveRecord, no alternative database adapters are supported
  • Opaque path structure: With paperclip and Carrierwave you can “design” your image path like you want, which might be good for long-term archival. AS does not provide such a functionality. The files are opaque without the corresponding ActiveStorage Blob table entry and are just a bunch of random file names.
  • Danger of N+1 queries. Make sure to call lists with a predefined scope: Organisation.with_attached_<attachment_name>.each ...
  • The whole app must use only one ActiveStorage service. It is not possible that one model uses a S3 bucket, the other one a Azure container etc. There is only a mirror option.

Pros of ActiveStorage

  • Upcoming standard solution, paperclip already got deprecated
  • No migration needed when adding new attachments
  • More secure by default - All attachments are always go through a Rails action that generates an access link with expiration. No files in public dir.
  • Versions are generated on demand and can be tuned until fit
  • Concept of previewers to generate thumbnails of pdfs, videos