1. 27 Feb, 2019 1 commit
  2. 25 Jan, 2019 2 commits
    • Gabriel Mazetto's avatar
      Refactor Storage Migration · 7bc16889
      Gabriel Mazetto authored
      Specs were reviewed and improved to better cover the current behavior.
      There was some standardization done as well to facilitate the
      implementation of the rollback functionality.
      
      StorageMigratorWorker was extracted to HashedStorage namespace were
      RollbackerWorker will live one as well.
      7bc16889
    • Gabriel Mazetto's avatar
      Prepare rake task for storage rollback · c2c34eba
      Gabriel Mazetto authored
      We are keeping compatibility with existing scheduled jobs.
      c2c34eba
  3. 19 Dec, 2018 1 commit
  4. 13 Dec, 2018 1 commit
    • Zeger-Jan van de Weg's avatar
      Restore Object Pools when restoring an object pool · 89a407dc
      Zeger-Jan van de Weg authored
      Pool repositories are persisted in the database, and when the DB is
      restored, the data need to be restored on disk. This is done by
      resetting the state machine and rescheduling the object pool creation.
      
      This is not an exact replica of the state like at the time of the
      creation of the backup. However, the data is consistent again.
      
      Dumping isn't required as internally GitLab uses git bundles which
      bundle all refs and include all objects in the bundle that they require,
      reduplicating as more repositories get backed up. This does require more
      data to be stored.
      
      Fixes https://gitlab.com/gitlab-org/gitaly/issues/1355
      89a407dc
  5. 06 Dec, 2018 1 commit
  6. 03 Dec, 2018 1 commit
  7. 26 Nov, 2018 2 commits
  8. 07 Nov, 2018 1 commit
    • Toon Claes's avatar
      Enhance performance of counting local Uploads · 1c481b7a
      Toon Claes authored
      Add an index to the `store` column on `uploads`. This makes counting
      local uploads faster.
      
      Also, there is no longer need to check for objects with `store = NULL`.
      See https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18557
      
      ---
      
      ### Query plans
      
      Query:
      
      ```sql
      SELECT COUNT(*)
      FROM "uploads"
      WHERE ("uploads"."store" = ? OR "uploads"."store" IS NULL)
      ```
      
      #### Without index
      
      ```
      gitlabhq_production=# EXPLAIN ANALYZE SELECT uploads.* FROM uploads WHERE (uploads.store = 1 OR uploads.store IS NULL);
                                                        QUERY PLAN
      ---------------------------------------------------------------------------------------------------------------
       Seq Scan on uploads  (cost=0.00..601729.54 rows=578 width=272) (actual time=6.170..2308.256 rows=545 loops=1)
         Filter: ((store = 1) OR (store IS NULL))
         Rows Removed by Filter: 4411957
       Planning time: 38.652 ms
       Execution time: 2308.454 ms
      (5 rows)
      ```
      
      #### Add index
      
      ```
      gitlabhq_production=# create index uploads_tmp1 on uploads (store);
      CREATE INDEX
      ```
      
      #### With index
      
      ```
      gitlabhq_production=# EXPLAIN ANALYZE SELECT uploads.* FROM uploads WHERE (uploads.store = 1 OR uploads.store IS NULL);
                                                                QUERY PLAN
      -------------------------------------------------------------------------------------------------------------------------------
       Bitmap Heap Scan on uploads  (cost=11.46..1238.88 rows=574 width=272) (actual time=0.155..0.577 rows=545 loops=1)
         Recheck Cond: ((store = 1) OR (store IS NULL))
         Heap Blocks: exact=217
         ->  BitmapOr  (cost=11.46..11.46 rows=574 width=0) (actual time=0.116..0.116 rows=0 loops=1)
               ->  Bitmap Index Scan on uploads_tmp1  (cost=0.00..8.74 rows=574 width=0) (actual time=0.095..0.095 rows=545 loops=1)
                     Index Cond: (store = 1)
               ->  Bitmap Index Scan on uploads_tmp1  (cost=0.00..2.44 rows=1 width=0) (actual time=0.020..0.020 rows=0 loops=1)
                     Index Cond: (store IS NULL)
       Planning time: 0.274 ms
       Execution time: 0.637 ms
      (10 rows)
      ```
      
      Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/6070
      1c481b7a
  9. 01 Nov, 2018 1 commit
  10. 25 Sep, 2018 1 commit
    • Valery Sizov's avatar
      Geo: sync disabled wikis. Stage 2 · 15d011d6
      Valery Sizov authored
      We started syncing all the wiki regardless of the fact it's disabled or
      not. We couldn't do that in one stage because of needing of smoth update
      and deprecating things. This is the second stage that finally removes
      unused columns in the geo_node_status table.
      15d011d6
  11. 13 Sep, 2018 1 commit
  12. 12 Sep, 2018 1 commit
  13. 07 Sep, 2018 1 commit
    • Zeger-Jan van de Weg's avatar
      Port cleanup tasks to use Gitaly · 3aedccb1
      Zeger-Jan van de Weg authored
      Rake tasks cleaning up the Git storage were still using direct disk
      access, which won't work if these aren't attached. To mitigate a
      migration issue was created.
      
      To port gitlab:cleanup:dirs, and gitlab:cleanup:repos, a new RPC was
      required, ListDirectories. This was implemented in Gitaly, through
      https://gitlab.com/gitlab-org/gitaly/merge_requests/868.
      
      To be able to use the new RPC the Gitaly server was bumped to v0.120.
      
      This is an RPC that will not use feature gates, as this doesn't scale on
      .com so there is no way to test it at scale. Futhermore, we _know_ it
      doesn't scale, but this might be a useful task for smaller instances.
      
      Lastly, the tests are slightly updated to also work when the disk isn't
      attached. Eventhough this is not planned, it was very little effort and
      thus I applied the boy scout rule.
      
      Closes https://gitlab.com/gitlab-org/gitaly/issues/954
      Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/40529
      3aedccb1
  14. 16 Aug, 2018 2 commits
  15. 12 Aug, 2018 1 commit
  16. 03 Aug, 2018 1 commit
  17. 31 Jul, 2018 3 commits
  18. 04 Jul, 2018 1 commit
    • Robert Speicher's avatar
      Speed up spec/tasks/gitlab/git_rake_spec.rb · f5da4815
      Robert Speicher authored
      Because no Git repository was actually created at the temporary path we
      were using, `git fsck` would traverse up until it found a repository,
      which in our case was the CE or EE repository.
      f5da4815
  19. 27 Jun, 2018 2 commits
  20. 15 Jun, 2018 1 commit
    • Will Chandler's avatar
      Add timestamps to gitlab-rake gitlab:backup:restore · cc41a771
      Will Chandler authored
      Adds a new method 'puts_time' that prepends the time of a
      message when printing it.  All instances of 'progress.puts'
      in the gitlab:backup:restore tasks are replaced with puts_time.
      Example output:
        2018-06-03 16:33:25 -0400 -- Restoring uploads ..
      Closes #46448
      cc41a771
  21. 12 Jun, 2018 1 commit
  22. 07 Jun, 2018 1 commit
  23. 05 Jun, 2018 1 commit
  24. 31 May, 2018 1 commit
  25. 29 May, 2018 1 commit
  26. 24 May, 2018 1 commit
  27. 25 Apr, 2018 1 commit
  28. 10 Apr, 2018 1 commit
  29. 02 Apr, 2018 1 commit
  30. 30 Mar, 2018 1 commit
  31. 29 Mar, 2018 2 commits
  32. 22 Mar, 2018 2 commits