• Stan Hu's avatar
    Expire project caches once per push instead of once per ref · f14647fd
    Stan Hu authored
    Previously `ProjectCacheWorker` would be scheduled once per ref, which
    would generate unnecessary I/O and load on Sidekiq, especially if many
    tags or branches were pushed at once. `ProjectCacheWorker` would expire
    three items:
    
    1. Repository size: This only needs to be updated once per push.
    2. Commit count: This only needs to be updated if the default branch
       is updated.
    3. Project method caches: This only needs to be updated if the default
       branch changes, but only if certain files change (e.g. README,
       CHANGELOG, etc.).
    
    Because the third item requires looking at the actual changes in the
    commit deltas, we schedule one `ProjectCacheWorker` to handle the first
    two cases, and schedule a separate `ProjectCacheWorker` for the third
    case if it is needed. As a result, this brings down the number of
    `ProjectCacheWorker` jobs from N to 2.
    
    Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52046
    f14647fd