These are all over 20 GB on GitLab.com. merge_request_diff_commits is several
hundred gigabytes in size.

This notifies developers when calling `find(_by!)` chained on
`execute`. And suggests using the methods from `FinderMethods`. These
will perform the correct authorization checks on the resource when it
is found.

{change_column_type,rename_column}_concurrently both copy data from one column
to another during a migration, which should not be done on GitLab.com. Instead,
we should use background migrations.

Signed-off-by: Rémy Coutable <remy@rymai.me>

Signed-off-by: Rémy Coutable <remy@rymai.me>

This reverts the addition of the "goldiloader" Gem and all use of it.
While this Gem is very promising it's causing a variety of problems on
GitLab.com due to it eager-loading too much data in places where we
don't expect/can handle this. At least for the time being this means we
have to go back to manually fixing N+1 query problems, but at least
those should not cause a negative impact on availability.

Goldiloader is great, but has several issues with has_many :through relations:

* https://github.com/salsify/goldiloader/issues/12
* https://github.com/salsify/goldiloader/issues/14
* https://github.com/salsify/goldiloader/issues/18

Rather than try to figure out which applies in each case, we should just do the
drudge work of manually disabling autoloading for all relations of this type. We
can always use regular preloading for specific cases, but this way we avoid
generating invalid queries through Goldiloader's magic.

Signed-off-by: Rémy Coutable <remy@rymai.me>

Server Side Request Forgery in Services and Web Hooks

See merge request gitlab/gitlabhq!2337

with StrongMemoize

Also try to use the same config from EE

Signed-off-by: Rémy Coutable <remy@rymai.me>

remove_column should only be used in the up (or change) step of a migration if
it's a post-deployment migration. Otherwise there will be downtime due to the
ActiveRecord column cache, which we can avoid by using the IgnorableColumn
concern in combination with a post-deployment migration.

And use .rubocop.yml to exclude paths we don't care,
rather than using the cop itself to exclude.

add_column_with_default is implemented in terms of update_column_in_batches, but
update_column_in_batches can be used independently. Neither of these should be
used on the specified large tables, because they will cause issues on large
instances like GitLab.com.

This also ignores the cop for all existing migrations, renaming
AddColumnWithDefaultToLargeTable where appropriate.

Adds a rubocop rule (with autocorrect) to ensure line break after guard clauses.

Signed-off-by: Rémy Coutable <remy@rymai.me>

Signed-off-by: Rémy Coutable <remy@rymai.me>

The types `timestamp` and `datetime` are aliases:
https://github.com/rails/rails/blob/v4.2.10/activerecord/lib/active_record/connection_adapters/abstract/schema_definitions.rb#L362-L364

Replaces all the explicit include metadata syntax in the specs (tag:
true) into the implicit one (:tag).
Added a cop to prevent future errors and handle autocorrection.

- ci_builds -- 33 million rows, 55 GB
- merge_request_diff_files -- 5 million rows, 9 GB (and growing rapidly)
- merge_request_diffs -- 5 million rows, 190 GB

These indexes are not recorded in the WAL (at least until PostgreSQL
10) and this isn't worth the minor performance improvement over btree
indexes.

This module provides a class method called `each_batch` that can be used
to iterate tables in batches in a more efficient way compared to Rails'
`in_batches` method. This commit also includes a RuboCop cop to
blacklist the use of `in_batches` in favour of this new method.

This module provides a class method called `each_batch` that can be used
to iterate tables in batches in a more efficient way compared to Rails'
`in_batches` method. This commit also includes a RuboCop cop to
blacklist the use of `in_batches` in favour of this new method.