From 61ed133fea80041b354c78b043cec72dd8644101 Mon Sep 17 00:00:00 2001 From: Eugen Rochko Date: Wed, 21 Feb 2018 23:21:32 +0100 Subject: Account archive download (#6460) * Fix #201: Account archive download * Export actor and private key in the archive * Optimize BackupService - Add conversation to cached associations of status, because somehow it was forgotten and is source of N+1 queries - Explicitly call GC between batches of records being fetched (Model class allocations are the worst offender) - Stream media files into the tar in 1MB chunks (Do not allocate media file (up to 8MB) as string into memory) - Use #bytesize instead of #size to calculate file size for JSON (Fix FileOverflow error) - Segment media into subfolders by status ID because apparently GIF-to-MP4 media are all named "media.mp4" for some reason * Keep uniquely generated filename in Paperclip::GifTranscoder * Ensure dumped files do not overwrite each other by maintaing directory partitions * Give tar archives a good name * Add scheduler to remove week-old backups * Fix code style issue --- config/locales/en.yml | 11 +++++++++++ config/routes.rb | 2 +- config/sidekiq.yml | 3 +++ 3 files changed, 15 insertions(+), 1 deletion(-) (limited to 'config') diff --git a/config/locales/en.yml b/config/locales/en.yml index 5cd3b08cf..b9dd5bd51 100644 --- a/config/locales/en.yml +++ b/config/locales/en.yml @@ -421,6 +421,13 @@ en: title: This page is not correct noscript_html: To use the Mastodon web application, please enable JavaScript. Alternatively, try one of the native apps for Mastodon for your platform. exports: + archive_takeout: + date: Date + download: Download your archive + hint_html: You can request an archive of your toots and uploaded media. The exported data will be in ActivityPub format, readable by any compliant software. + in_progress: Compiling your archive... + request: Request your archive + size: Size blocks: You block csv: CSV follows: You follow @@ -733,6 +740,10 @@ en: setup: Set up wrong_code: The entered code was invalid! Are server time and device time correct? user_mailer: + backup_ready: + explanation: You requested a full backup of your Mastodon account. It's now ready for download! + subject: Your archive is ready for download + title: Archive takeout welcome: edit_profile_action: Setup profile edit_profile_step: You can customize your profile by uploading an avatar, header, changing your display name and more. If you’d like to review new followers before they’re allowed to follow you, you can lock your account. diff --git a/config/routes.rb b/config/routes.rb index 9f541200a..7ed2d61f1 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -76,7 +76,7 @@ Rails.application.routes.draw do resource :notifications, only: [:show, :update] resource :import, only: [:show, :create] - resource :export, only: [:show] + resource :export, only: [:show, :create] namespace :exports, constraints: { format: :csv } do resources :follows, only: :index, controller: :following_accounts resources :blocks, only: :index, controller: :blocked_accounts diff --git a/config/sidekiq.yml b/config/sidekiq.yml index bfe29b8f8..244e9ea48 100644 --- a/config/sidekiq.yml +++ b/config/sidekiq.yml @@ -30,3 +30,6 @@ email_scheduler: cron: '0 10 * * 2' class: Scheduler::EmailScheduler + backup_cleanup_scheduler: + cron: '<%= Random.rand(0..59) %> <%= Random.rand(3..5) %> * * *' + class: Scheduler::BackupCleanupScheduler -- cgit