1. 05 Mar, 2018 1 commit
  2. 07 Feb, 2018 1 commit
  3. 12 Jan, 2018 1 commit
  4. 10 Jan, 2018 1 commit
    • Jan Provaznik's avatar
      Denormalize commits count for merge request diffs · e6a1db6d
      Jan Provaznik authored
      For each MR diff an extra 'SELECT COUNT()' is executed
      to get number of commits for the diff. Overall time to get counts for
      all MR diffs may be quite expensive. To speed up loading of MR info,
      information about number of commits is stored in a MR diff's extra column.
      Closes #38068
  5. 12 Dec, 2017 1 commit
    • Zeger-Jan van de Weg's avatar
      Use memoization for commits on diffs · 3ab026b7
      Zeger-Jan van de Weg authored
      The Gitaly CommitService is being hammered by n + 1 calls, mostly when
      finding commits. This leads to this gRPC being turned of on production:
      Hunting down where it came from, most of them were due to
      MergeRequest#show. To prove this, I set a script to request the
      MergeRequest#show page 50 times. The GDK was being scraped by
      Prometheus, where we have metrics on controller#action and their Gitaly
      calls performed. On both occations I've restarted the full GDK so all
      caches had to be rebuild.
      Current master, 806a68a8, needed 435 requests
      After this commit, 154 requests
  6. 28 Nov, 2017 1 commit
    • Sean McGivern's avatar
      Remove serialised diff and commit columns · 4ebbfe5d
      Sean McGivern authored
      The st_commits and st_diffs columns on merge_request_diffs historically held the
      YAML-serialised data for a merge request diff, in a variety of formats.
      Since 9.5, these have been migrated in the background to two new tables:
      merge_request_diff_commits and merge_request_diff_files. That has the advantage
      that we can actually query the data (for instance, to find out how many commits
      we've stored), and that it can't be in a variety of formats, but must match the
      new schema.
      This is the final step of that journey, where we drop those columns and remove
      all references to them. This is a breaking change to the importer, because we
      can no longer import diffs created in the old format, and we cannot guarantee
      the export will be in the new format unless it was generated after this commit.
  7. 23 Nov, 2017 1 commit
    • Sean McGivern's avatar
      Use latest_merge_request_diff association · 991bf24e
      Sean McGivern authored
      Compared to the merge_request_diff association:
      1. It's simpler to query. The query uses a foreign key to the
         merge_request_diffs table, so no ordering is necessary.
      2. It's faster for preloading. The merge_request_diff association has to load
         every diff for the MRs in the set, then discard all but the most recent for
         each. This association means that Rails can just query for N diffs from N
      3. It's more complicated to update. This is a bidirectional foreign key, so we
         need to update two tables when adding a diff record. This also means we need
         to handle this as a special case when importing a GitLab project.
      There is some juggling with this association in the merge request model:
      * `MergeRequest#latest_merge_request_diff` is _always_ the latest diff.
      * `MergeRequest#merge_request_diff` reuses
        `MergeRequest#latest_merge_request_diff` unless:
          * Arguments are passed. These are typically to force-reload the association.
          * It doesn't exist. That means we might be trying to implicitly create a
            diff. This only seems to happen in specs.
          * The association is already loaded. This is important for the reasons
            explained in the comment, which I'll reiterate here: if we a) load a
            non-latest diff, then b) get its `merge_request`, then c) get that MR's
            `merge_request_diff`, we should get the diff we loaded in c), even though
            that's not the latest diff.
      Basically, `MergeRequest#merge_request_diff` is the latest diff in most cases,
      but not quite all.
  8. 16 Nov, 2017 1 commit
  9. 02 Nov, 2017 1 commit
    • Sean McGivern's avatar
      Set merge_request_diff_id on MR when creating · 4768a1e2
      Sean McGivern authored
      Once we migrate existing MRs to have this column, we will be able to get the
      latest diff for a single merge request more efficiently, and (more importantly)
      get all latest diffs for a collection of MRs efficiently.
  10. 05 Oct, 2017 3 commits
  11. 08 Aug, 2017 1 commit
  12. 03 Aug, 2017 1 commit
    • Sean McGivern's avatar
      Migrate MR commits and diffs to new tables · f2d50af9
      Sean McGivern authored
      Previously, we stored these as serialised fields - `st_{commits,diffs}` - on the
      `merge_request_diffs` table. These now have their own tables -
      `merge_request_diff_{commits,diffs}` - with a column for each attribute of the
      serialised data.
      Add a background migration to go through the existing MR diffs and migrate them
      to the new format. Ignore any contents that cannot be displayed. Assuming that
      we have 5 million rows to migrate, and each batch of 2,500 rows can be
      completed in 5 minutes, this will take about 7 days to migrate everything.
  13. 26 Jul, 2017 1 commit
    • Sean McGivern's avatar
      Fix saving diffs that are not valid UTF-8 · 396b8f91
      Sean McGivern authored
      Previously, we used Psych, which would:
      1. Check if a string was encoded as binary, and not ASCII-compatible.
      2. Add the !binary tag in that case.
      3. Convert to base64.
      We need to do the same thing, using a new column in place of the tag.
  14. 06 Jul, 2017 2 commits
    • Sean McGivern's avatar
      Add table for merge request commits · aff5c9f3
      Sean McGivern authored
      This is an ID-less table with just three columns: an association to the merge
      request diff the commit belongs to, the relative order of the commit within the
      merge request diff, and the commit SHA itself.
      Previously we stored much more information about the commits, so that we could
      display them even when they were deleted from the repo. Since 8.0, we ensure
      that those commits are kept around for as long as the target repo itself is, so
      we don't need to duplicate that data in the database.
    • Yorick Peterse's avatar
      Rename ActiverecordSerialize cop · e1a3bf30
      Yorick Peterse authored
      This cop has been renamed to ActiveRecordSerialize to match the way
      "ActiveRecord" is usually written.
  15. 16 Jun, 2017 1 commit
    • Sean McGivern's avatar
      Add table for files in merge request diffs · 9a73b634
      Sean McGivern authored
      This adds an ID-less table containing one row per file, per merge request
      diff. It has a column for each attribute on Gitlab::Git::Diff that is serialised
      currently, with the advantage that we can easily query the attributes of this
      new table.
      It does not migrate existing data, so we have fallback code when the legacy
      st_diffs column is present instead. For a merge request diff to be valid, it
      should have at most one of:
      * Rows in this new table, with the correct merge_request_diff_id.
      * A non-NULL st_diffs column.
      It may have neither, if the diff is empty.
  16. 01 Jun, 2017 1 commit
  17. 31 May, 2017 1 commit
  18. 23 May, 2017 4 commits
  19. 25 Apr, 2017 1 commit
    • Sean McGivern's avatar
      Show correct size when MR diff overflows · a0979c05
      Sean McGivern authored
      The problem is that we often go via a diff object constructed from the diffs
      stored in the DB. Those diffs, by definition, don't overflow, so we don't have
      access to the 'correct' `real_size` - that is stored on the MR diff object
  20. 11 Apr, 2017 1 commit
  21. 10 Apr, 2017 1 commit
  22. 08 Apr, 2017 3 commits
  23. 04 Apr, 2017 1 commit
  24. 23 Feb, 2017 5 commits
  25. 12 Jan, 2017 1 commit
  26. 05 Jan, 2017 2 commits
  27. 08 Dec, 2016 1 commit