GitLab

Add option to whitelist _html fields from attributes on CacheMarkdownField

See merge request gitlab-org/gitlab-ce!26374

(cherry picked from commit f7fcfc77)

3fd8e612 Add option to not exclude _html fields from attributes
bcc988a6 Does not exclude message_html from attributes
69dc893d Fix spec for Gitlab::JsonCache
6264694d Rename the hidden option to whitelisted

Cached markdown version is composed both from global and local
markdown version. This allows admins to bump version locally when
needed (e.g. when external URL is changed).

This engine was replaced with CommonMarker in 11.4, it was deprecated
since then.

Since we needed to bump the version to 13 in the backports,
and we know that an MR on master also bumped it
to 13, bump to 14 to ensure that when a customer
upgrades to the most recent release, the markdown
gets recalcuated as necessary.

Since we needed to bump the version to 13 in the backports,
and we know that an MR on master also bumped it
to 13, bump to 14 to ensure that when a customer
upgrades to the most recent release, the markdown
gets recalcuated as necessary.

Since the rendering of footnotes is changed, the cached
html need to be updated

[master] XSS in markdown following unrecognized HTML element

Closes #2732

See merge request gitlab/gitlabhq!2599

Partially addresses #47424.

and also make sure that if a field is saved, then the existing cache version is maintained or only upgraded to the version with the same markdown engine.

Treat empty markdown and html strings as valid cached text, not missing cache that needs to be updated

In GitLab EE, a GitLab instance can be read-only (e.g. when it's a Geo
secondary node). But in GitLab CE it also might be useful to have the
"read-only" idea around. So port it back to GitLab CE.

Also having the principle of read-only in GitLab CE would hopefully
lead to less errors introduced, doing write operations when there
aren't allowed for read-only calls.

Closes gitlab-org/gitlab-ce#37534.

An upcoming update to rubocop-gitlab-security added additional
violations.

If the cached html_field for a markdown_field is `nil` while the
mfarkdown_field is not, it needs to be refreshed.

Swap a before_save call with a before_create/before_update pair to avoid a confict with elasticsearch-model

This commit adds a number of _html columns and, with the exception of Note,
starts updating them whenever the content of their partner fields changes.

Note has a collision with the note_html attr_accessor; that will be fixed later

A background worker for clearing these cache columns is also introduced - use
`rake cache:clear` to set it off. You can clear the database or Redis caches
separately by running `rake cache:clear:db` or `rake cache:clear:redis`,
respectively.