- we now use the hierarchy class also for epics
- also rename supports_nested_groups? into supports_nested_objects?
  - move it to a concern

Adds a background migration that will ensure all projects that
are on legacy storage have a row in `project_repositories`.

This change the write to accept an options hash to make
it compatible with ActiveSupport::Cache::Store#write method.

The options hash are not passed to the underlying cache
implementation because RequestStore#write accepts only
a key, and value params.

This upgrade moves logic from gitlab-rails to Gitaly, which allowed code
removal too.

This reverts commit 00acef43.

When the BFG object map file is in object storage (i.e., uploads in
general are placed into object storage), we get an instance of the
Gitlab::HttpIO class. This doesn't behave as expected when you try to
read past EOF, so we need to explicitly check for this condition to
avoid ending up in a tight loop around io.read

This commit adds a name to each release, defaulting it to tag name,
keeps track of the SHA when a new release is created and tracks the
current user as release author.

spec/lib/gitlab/background_migration/migrate_stage_status_spec.rb:9: warning: already initialized constant STATUSES
spec/lib/gitlab/background_migration/migrate_build_stage_spec.rb:9: warning: previous definition of STATUSES was here

It adds a base class for CompareTestReportsService
containing common code with CompareLicenseManagementReportsService
which is present in GitLab Enterprise Edition.

Signed-off-by: Rémy Coutable <remy@rymai.me>

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/55079

When a project is forked, the new repository used to be a deep copy of everything
stored on disk by leveraging `git clone`. This works well, and makes isolation
between repository easy. However, the clone is at the start 100% the same as the
origin repository. And in the case of the objects in the object directory, this
is almost always going to be a lot of duplication.

Object Pools are a way to create a third repository that essentially only exists
for its 'objects' subdirectory. This third repository's object directory will be
set as alternate location for objects. This means that in the case an object is
missing in the local repository, git will look in another location. This other
location is the object pool repository.

When Git performs garbage collection, it's smart enough to check the
alternate location. When objects are duplicated, it will allow git to
throw one copy away. This copy is on the local repository, where to pool
remains as is.

These pools have an origin location, which for now will always be a
repository that itself is not a fork. When the root of a fork network is
forked by a user, the fork still clones the full repository. Async, the
pool repository will be created.

Either one of these processes can be done earlier than the other. To
handle this race condition, the Join ObjectPool operation is
idempotent. Given its idempotent, we can schedule it twice, with the
same effect.

To accommodate the holding of state two migrations have been added.
1. Added a state column to the pool_repositories column. This column is
managed by the state machine, allowing for hooks on transitions.
2. pool_repositories now has a source_project_id. This column in
convenient to have for multiple reasons: it has a unique index allowing
the database to handle race conditions when creating a new record. Also,
it's nice to know who the host is. As that's a short link to the fork
networks root.

Object pools are only available for public project, which use hashed
storage and when forking from the root of the fork network. (That is,
the project being forked from itself isn't a fork)

In this commit message I use both ObjectPool and Pool repositories,
which are alike, but different from each other. ObjectPool refers to
whatever is on the disk stored and managed by Gitaly. PoolRepository is
the record in the database.

If a commit has two parents, one is direct ancestor, and one is not,
and the order of `commits` is in such fashion that the non-ancestor
side is visited first, the commit would be determined as non-ancestor,
when in fact it can be.

Therefore we should first determine all direct ancestors
prior to analyzing.

Analyze new commits graph to determine each commit's merge commit.

Fix "merged with [commit]" info for merge requests being merged
automatically by other actions.

Allow analyzing upto the relevant commit

This adds a background migration that will ensure all projects that
are on hashed storage have a row in `project_repositories`.

Related issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/48527

It gathers list of file paths to delete before destroying
the parent object. Then after the parent_object is destroyed
these paths are scheduled for deletion asynchronously.

Carrierwave needed associated model for deleting upload file.
To avoid this requirement, simple Fog/File layer is used directly
for file deletion, this allows us to use just a simple list of paths.

This reverts commit 3560b119.

This changes `correlation_id` to be `correlation-id` when passed via jobs

The Correlation ID is taken or generated from received X-Request-ID.
Then it is being passed to all executed services (sidekiq workers
or gitaly calls).

The Correlation ID is logged in all structured logs as `correlation_id`.

Restricts unicode characters and IDNA deviations
which could be used in a phishing attack

* removed 100 limit on file search results because we
  load all results anyway
* expensive processing (parsing match content, utf encoding)
  is done only for selected page in paginated output

An unknown public GPG key will result in a GPGME::Error thrown from gpg,
which would cause an Error 500 on the signatures endpoint.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/54729

The API permits path traversal characters like '../' to be passed down
to the template finder. Detect these requests and cause them to fail
with a 500 response code.

This reverts commit 22954f22