GitLab steht aufgrund wichtiger Wartungsarbeiten am Montag, den 8. März, zwischen 17:00 und 19:00 Uhr nicht zur Verfügung.

Commit 4a2320a7 authored by Douwe Maan's avatar Douwe Maan

Merge branch 'new-issue-by-email' into 'master'

Implement #3243 New Issue by email

So we extend Gitlab::Email::Receiver for this new behaviour,
however we might want to split it into another class for better
testing it.

Another issue is that, currently it's using this to parse project
identifier:

    Gitlab::IncomingEmail.key_from_address

Which is using:

    Gitlab.config.incoming_email.address

for the receiver name. This is probably `reply` because it's used
for replying to a specific issue. We might want to introduce another
config for this, or just use `reply` instead of `incoming`.

I'll prefer to introduce a new config for this, or just change
`reply` to `incoming` because it would make sense for replying to
there, too.

The email template used in tests were copied and modified from:
`emails/valid_reply.eml` which I hope is ok.

/cc @DouweM #3243

See merge request !3363
parents b24ccd4a 9370bb23
......@@ -13,6 +13,7 @@ v 8.11.0 (unreleased)
- Retrieve rendered HTML from cache in one request
- Fix renaming repository when name contains invalid chararacters under project settings
- Nokogiri's various parsing methods are now instrumented
- Add a way to send an email and create an issue based on private personal token. Find the email address from issues page. !3363
- Add build event color in HipChat messages (David Eisner)
- Make fork counter always clickable. !5463 (winniehell)
- All created issues, API or WebUI, can be submitted to Akismet for spam check !5333
......
......@@ -99,3 +99,33 @@ form.edit-issue {
.issue-form .select2-container {
width: 250px !important;
}
.issues-footer {
padding-top: $gl-padding;
padding-bottom: 37px;
}
.issue-email-modal-btn {
padding: 0;
color: $gl-link-color;
background-color: transparent;
border: 0;
outline: 0;
&:hover {
text-decoration: underline;
}
}
.email-modal-input-group {
margin-bottom: 10px;
.form-control {
background-color: $white-light;
}
.btn {
background-color: $background-color;
border: 1px solid $border-gray-light;
}
}
......@@ -331,7 +331,7 @@ def token
end
def valid_token?(token)
project.valid_runners_token? token
project.valid_runners_token?(token)
end
def has_tags?
......
......@@ -605,6 +605,13 @@ def web_url_without_protocol
web_url.split('://')[1]
end
def new_issue_address(author)
if Gitlab::IncomingEmail.enabled? && author
Gitlab::IncomingEmail.reply_address(
"#{path_with_namespace}+#{author.authentication_token}")
end
end
def build_commit_note(commit)
notes.new(commit_id: commit.id, noteable_type: 'Commit')
end
......
.issues-footer.text-center
%button.issue-email-modal-btn{ type: "button", data: { toggle: "modal", target: "#issue-email-modal" } }
Email a new issue to this project
#issue-email-modal.modal.fade{ tabindex: "-1", role: "dialog" }
.modal-dialog{ role: "document" }
.modal-content
.modal-header
%button.close{ type: "button", data: { dismiss: "modal" }, aria: { label: "close" } }
%span{ aria: { hidden: "true" } }= icon("times")
%h4.modal-title
Create new issue by email
.modal-body
%p
Write an email to the below email address. (This is a private email address, so keep it secret.)
.email-modal-input-group.input-group
= text_field_tag :issue_email, email, class: "monospace js-select-on-focus form-control", readonly: true
.input-group-btn
= clipboard_button(clipboard_target: '#issue_email')
%p
Send an email to this address to create an issue.
%p
Use the subject line as the title of your issue.
%p
Use the message as the body of your issue (feel free to include some nice
= succeed ")." do
= link_to "Markdown", help_page_path('markdown', 'markdown')
- @no_container = true
- page_title "Issues"
- new_issue_email = @project.new_issue_address(current_user)
= render "projects/issues/head"
= content_for :meta_tags do
......@@ -23,7 +24,9 @@
= render 'shared/issuable/filter', type: :issues
.issues-holder
= render "issues"
= render 'issues'
- if new_issue_email
= render 'issue_by_email', email: new_issue_email
- else
.blank-state.blank-state-welcome
%h2.blank-state-title.blank-state-welcome-title
......@@ -40,3 +43,5 @@
- if can? current_user, :create_issue, @project
= link_to new_namespace_project_issue_path(@project.namespace, @project), class: "btn btn-new", title: "New Issue", id: "new_issue_link" do
New Issue
- if new_issue_email
= render 'issue_by_email', email: new_issue_email
......@@ -21,31 +21,35 @@ def handle_failure(raw, e)
return unless raw.present?
can_retry = false
reason = nil
case e
when Gitlab::Email::Receiver::SentNotificationNotFoundError
reason = "We couldn't figure out what the email is in reply to. Please create your comment through the web interface."
when Gitlab::Email::Receiver::EmptyEmailError
can_retry = true
reason = "It appears that the email is blank. Make sure your reply is at the top of the email, we can't process inline replies."
when Gitlab::Email::Receiver::AutoGeneratedEmailError
reason = "The email was marked as 'auto generated', which we can't accept. Please create your comment through the web interface."
when Gitlab::Email::Receiver::UserNotFoundError
reason = "We couldn't figure out what user corresponds to the email. Please create your comment through the web interface."
when Gitlab::Email::Receiver::UserBlockedError
reason = "Your account has been blocked. If you believe this is in error, contact a staff member."
when Gitlab::Email::Receiver::UserNotAuthorizedError
reason = "You are not allowed to respond to the thread you are replying to. If you believe this is in error, contact a staff member."
when Gitlab::Email::Receiver::NoteableNotFoundError
reason = "The thread you are replying to no longer exists, perhaps it was deleted? If you believe this is in error, contact a staff member."
when Gitlab::Email::Receiver::InvalidNoteError
can_retry = true
reason = e.message
else
return
reason =
case e
when Gitlab::Email::UnknownIncomingEmail
"We couldn't figure out what the email is for. Please create your issue or comment through the web interface."
when Gitlab::Email::SentNotificationNotFoundError
"We couldn't figure out what the email is in reply to. Please create your comment through the web interface."
when Gitlab::Email::ProjectNotFound
"We couldn't find the project. Please check if there's any typo."
when Gitlab::Email::EmptyEmailError
can_retry = true
"It appears that the email is blank. Make sure your reply is at the top of the email, we can't process inline replies."
when Gitlab::Email::AutoGeneratedEmailError
"The email was marked as 'auto generated', which we can't accept. Please create your comment through the web interface."
when Gitlab::Email::UserNotFoundError
"We couldn't figure out what user corresponds to the email. Please create your comment through the web interface."
when Gitlab::Email::UserBlockedError
"Your account has been blocked. If you believe this is in error, contact a staff member."
when Gitlab::Email::UserNotAuthorizedError
"You are not allowed to perform this action. If you believe this is in error, contact a staff member."
when Gitlab::Email::NoteableNotFoundError
"The thread you are replying to no longer exists, perhaps it was deleted? If you believe this is in error, contact a staff member."
when Gitlab::Email::InvalidNoteError,
Gitlab::Email::InvalidIssueError
can_retry = true
e.message
end
if reason
EmailRejectionMailer.rejection(reason, raw, can_retry).deliver_later
end
EmailRejectionMailer.rejection(reason, raw, can_retry).deliver_later
end
end
require 'gitlab/email/handler/create_note_handler'
require 'gitlab/email/handler/create_issue_handler'
module Gitlab
module Email
module Handler
HANDLERS = [CreateNoteHandler, CreateIssueHandler]
def self.for(mail, mail_key)
HANDLERS.find do |klass|
handler = klass.new(mail, mail_key)
break handler if handler.can_handle?
end
end
end
end
end
module Gitlab
module Email
module Handler
class BaseHandler
attr_reader :mail, :mail_key
def initialize(mail, mail_key)
@mail = mail
@mail_key = mail_key
end
def message
@message ||= process_message
end
def author
raise NotImplementedError
end
def project
raise NotImplementedError
end
private
def validate_permission!(permission)
raise UserNotFoundError unless author
raise UserBlockedError if author.blocked?
raise ProjectNotFound unless author.can?(:read_project, project)
raise UserNotAuthorizedError unless author.can?(permission, project)
end
def process_message
message = ReplyParser.new(mail).execute.strip
add_attachments(message)
end
def add_attachments(reply)
attachments = Email::AttachmentUploader.new(mail).execute(project)
reply + attachments.map do |link|
"\n\n#{link[:markdown]}"
end.join
end
def verify_record!(record:, invalid_exception:, record_name:)
return if record.persisted?
error_title = "The #{record_name} could not be created for the following reasons:"
msg = error_title + record.errors.full_messages.map do |error|
"\n\n- #{error}"
end.join
raise invalid_exception, msg
end
end
end
end
end
require 'gitlab/email/handler/base_handler'
module Gitlab
module Email
module Handler
class CreateIssueHandler < BaseHandler
attr_reader :project_path, :authentication_token
def initialize(mail, mail_key)
super(mail, mail_key)
@project_path, @authentication_token =
mail_key && mail_key.split('+', 2)
end
def can_handle?
!authentication_token.nil?
end
def execute
raise ProjectNotFound unless project
validate_permission!(:create_issue)
verify_record!(
record: create_issue,
invalid_exception: InvalidIssueError,
record_name: 'issue')
end
def author
@author ||= User.find_by(authentication_token: authentication_token)
end
def project
@project ||= Project.find_with_namespace(project_path)
end
private
def create_issue
Issues::CreateService.new(
project,
author,
title: mail.subject,
description: message
).execute
end
end
end
end
end
require 'gitlab/email/handler/base_handler'
module Gitlab
module Email
module Handler
class CreateNoteHandler < BaseHandler
def can_handle?
mail_key =~ /\A\w+\z/
end
def execute
raise SentNotificationNotFoundError unless sent_notification
raise AutoGeneratedEmailError if mail.header.to_s =~ /auto-(generated|replied)/
validate_permission!(:create_note)
raise NoteableNotFoundError unless sent_notification.noteable
raise EmptyEmailError if message.blank?
verify_record!(
record: create_note,
invalid_exception: InvalidNoteError,
record_name: 'comment')
end
def author
sent_notification.recipient
end
def project
sent_notification.project
end
def sent_notification
@sent_notification ||= SentNotification.for(mail_key)
end
private
def create_note
Notes::CreateService.new(
project,
author,
note: message,
noteable_type: sent_notification.noteable_type,
noteable_id: sent_notification.noteable_id,
commit_id: sent_notification.commit_id,
line_code: sent_notification.line_code
).execute
end
end
end
end
end
require 'gitlab/email/handler'
# Inspired in great part by Discourse's Email::Receiver
module Gitlab
module Email
class Receiver
class ProcessingError < StandardError; end
class EmailUnparsableError < ProcessingError; end
class SentNotificationNotFoundError < ProcessingError; end
class EmptyEmailError < ProcessingError; end
class AutoGeneratedEmailError < ProcessingError; end
class UserNotFoundError < ProcessingError; end
class UserBlockedError < ProcessingError; end
class UserNotAuthorizedError < ProcessingError; end
class NoteableNotFoundError < ProcessingError; end
class InvalidNoteError < ProcessingError; end
class ProcessingError < StandardError; end
class EmailUnparsableError < ProcessingError; end
class SentNotificationNotFoundError < ProcessingError; end
class ProjectNotFound < ProcessingError; end
class EmptyEmailError < ProcessingError; end
class AutoGeneratedEmailError < ProcessingError; end
class UserNotFoundError < ProcessingError; end
class UserBlockedError < ProcessingError; end
class UserNotAuthorizedError < ProcessingError; end
class NoteableNotFoundError < ProcessingError; end
class InvalidNoteError < ProcessingError; end
class InvalidIssueError < ProcessingError; end
class UnknownIncomingEmail < ProcessingError; end
class Receiver
def initialize(raw)
@raw = raw
end
......@@ -20,91 +26,38 @@ def initialize(raw)
def execute
raise EmptyEmailError if @raw.blank?
raise SentNotificationNotFoundError unless sent_notification
raise AutoGeneratedEmailError if message.header.to_s =~ /auto-(generated|replied)/
author = sent_notification.recipient
raise UserNotFoundError unless author
raise UserBlockedError if author.blocked?
project = sent_notification.project
raise UserNotAuthorizedError unless project && author.can?(:create_note, project)
raise NoteableNotFoundError unless sent_notification.noteable
reply = ReplyParser.new(message).execute.strip
raise EmptyEmailError if reply.blank?
reply = add_attachments(reply)
note = create_note(reply)
mail = build_mail
mail_key = extract_mail_key(mail)
handler = Handler.for(mail, mail_key)
unless note.persisted?
msg = "The comment could not be created for the following reasons:"
note.errors.full_messages.each do |error|
msg << "\n\n- #{error}"
end
raise UnknownIncomingEmail unless handler
raise InvalidNoteError, msg
end
handler.execute
end
private
def message
@message ||= Mail::Message.new(@raw)
rescue Encoding::UndefinedConversionError, Encoding::InvalidByteSequenceError => e
def build_mail
Mail::Message.new(@raw)
rescue Encoding::UndefinedConversionError,
Encoding::InvalidByteSequenceError => e
raise EmailUnparsableError, e
end
def reply_key
key_from_to_header || key_from_additional_headers
def extract_mail_key(mail)
key_from_to_header(mail) || key_from_additional_headers(mail)
end
def key_from_to_header
key = nil
message.to.each do |address|
def key_from_to_header(mail)
mail.to.find do |address|
key = Gitlab::IncomingEmail.key_from_address(address)
break if key
break key if key
end
key
end
def key_from_additional_headers
reply_key = nil
Array(message.references).each do |message_id|
reply_key = Gitlab::IncomingEmail.key_from_fallback_reply_message_id(message_id)
break if reply_key
def key_from_additional_headers(mail)
Array(mail.references).find do |mail_id|
key = Gitlab::IncomingEmail.key_from_fallback_message_id(mail_id)
break key if key
end
reply_key
end
def sent_notification
return nil unless reply_key
SentNotification.for(reply_key)
end
def add_attachments(reply)
attachments = Email::AttachmentUploader.new(message).execute(sent_notification.project)
attachments.each do |link|
reply << "\n\n#{link[:markdown]}"
end
reply
end
def create_note(reply)
sent_notification.create_note(reply)
end
end
end
......
module Gitlab
module IncomingEmail
class << self
FALLBACK_REPLY_MESSAGE_ID_REGEX = /\Areply\-(.+)@#{Gitlab.config.gitlab.host}\Z/.freeze
FALLBACK_MESSAGE_ID_REGEX = /\Areply\-(.+)@#{Gitlab.config.gitlab.host}\Z/.freeze
def enabled?
config.enabled && config.address
......@@ -21,8 +21,8 @@ def key_from_address(address)
match[1]
end
def key_from_fallback_reply_message_id(message_id)
match = message_id.match(FALLBACK_REPLY_MESSAGE_ID_REGEX)
def key_from_fallback_message_id(mail_id)
match = mail_id.match(FALLBACK_MESSAGE_ID_REGEX)
return unless match
match[1]
......
......@@ -524,6 +524,35 @@
end
end
describe 'new issue by email' do
shared_examples 'show the email in the modal' do
before do
stub_incoming_email_setting(enabled: true, address: "p+%{key}@gl.ab")
visit namespace_project_issues_path(project.namespace, project)
click_button('Email a new issue')
end
it 'click the button to show modal for the new email' do
page.within '#issue-email-modal' do
email = project.new_issue_address(@user)
expect(page).to have_selector("input[value='#{email}']")
end
end
end
context 'with existing issues' do
let!(:issue) { create(:issue, project: project, author: @user) }
it_behaves_like 'show the email in the modal'
end
context 'without existing issues' do
it_behaves_like 'show the email in the modal'
end
end
describe 'due date' do
context 'update due on issue#show', js: true do
let(:issue) { create(:issue, project: project, author: @user, assignee: @user) }
......
Return-Path: <jake@adventuretime.ooo>
Received: from iceking.adventuretime.ooo ([unix socket]) by iceking (Cyrus v2.2.13-Debian-2.2.13-19+squeeze3) with LMTPA; Thu, 13 Jun 2013 17:03:50 -0400
Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) by iceking.adventuretime.ooo (8.14.3/8.14.3/Debian-9.4) with ESMTP id r5DL3nFJ016967 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for <incoming+gitlabhq/gitlabhq@appmail.adventuretime.ooo>; Thu, 13 Jun 2013 17:03:50 -0400
Received: by mail-ie0-f180.google.com with SMTP id f4so21977375iea.25 for <incoming+gitlabhq/gitlabhq@appmail.adventuretime.ooo>; Thu, 13 Jun 2013 14:03:48 -0700
Received: by 10.0.0.1 with HTTP; Thu, 13 Jun 2013 14:03:48 -0700
Date: Thu, 13 Jun 2013 17:03:48 -0400
From: Jake the Dog <jake@adventuretime.ooo>
To: incoming+gitlabhq/gitlabhq+auth_token@appmail.adventuretime.ooo
Message-ID: <CADkmRc+rNGAGGbV2iE5p918UVy4UyJqVcXRO2=otppgzduJSg@mail.gmail.com>
Subject: New Issue by email
Mime-Version: 1.0
Content-Type: text/plain;
charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Sieve: CMU Sieve 2.2
X-Received: by 10.0.0.1 with SMTP id n7mr11234144ipb.85.1371157428600; Thu,
13 Jun 2013 14:03:48 -0700 (PDT)
X-Scanned-By: MIMEDefang 2.69 on IPv6:2001:470:1d:165::1
The reply by email functionality should be extended to allow creating a new issue by email.
* Allow an admin to specify which project the issue should be created under by checking the sender domain.
* Possibly allow the use of regular expression matches within the subject/body to specify which project the issue should be created under.
Return-Path: <jake@adventuretime.ooo>
Received: from iceking.adventuretime.ooo ([unix socket]) by iceking (Cyrus v2.2.13-Debian-2.2.13-19+squeeze3) with LMTPA; Thu, 13 Jun 2013 17:03:50 -0400
Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) by iceking.adventuretime.ooo (8.14.3/8.14.3/Debian-9.4) with ESMTP id r5DL3nFJ016967 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for <incoming+gitlabhq/gitlabhq@appmail.adventuretime.ooo>; Thu, 13 Jun 2013 17:03:50 -0400
Received: by mail-ie0-f180.google.com with SMTP id f4so21977375iea.25 for <incoming+gitlabhq/gitlabhq@appmail.adventuretime.ooo>; Thu, 13 Jun 2013 14:03:48 -0700
Received: by 10.0.0.1 with HTTP; Thu, 13 Jun 2013 14:03:48 -0700
Date: Thu, 13 Jun 2013 17:03:48 -0400
From: Jake the Dog <jake@adventuretime.ooo>
To: incoming+gitlabhq/gitlabhq+auth_token@appmail.adventuretime.ooo
Message-ID: <CADkmRc+rNGAGGbV2iE5p918UVy4UyJqVcXRO2=otppgzduJSg@mail.gmail.com>
Subject: New Issue by email
Mime-Version: 1.0
Content-Type: text/plain;
charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Sieve: CMU Sieve 2.2
X-Received: by 10.0.0.1 with SMTP id n7mr11234144ipb.85.1371157428600; Thu,
13 Jun 2013 14:03:48 -0700 (PDT)
X-Scanned-By: MIMEDefang 2.69 on IPv6:2001:470:1d:165::1
Return-Path: <jake@adventuretime.ooo>
Received: from iceking.adventuretime.ooo ([unix socket]) by iceking (Cyrus v2.2.13-Debian-2.2.13-19+squeeze3) with LMTPA; Thu, 13 Jun 2013 17:03:50 -0400
Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) by iceking.adventuretime.ooo (8.14.3/8.14.3/Debian-9.4) with ESMTP id r5DL3nFJ016967 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for <incoming+gitlabhq/gitlabhq@appmail.adventuretime.ooo>; Thu, 13 Jun 2013 17:03:50 -0400
Received: by mail-ie0-f180.google.com with SMTP id f4so21977375iea.25 for <incoming+gitlabhq/gitlabhq@appmail.adventuretime.ooo>; Thu, 13 Jun 2013 14:03:48 -0700
Received: by 10.0.0.1 with HTTP; Thu, 13 Jun 2013 14:03:48 -0700
Date: Thu, 13 Jun 2013 17:03:48 -0400
From: Jake the Dog <jake@adventuretime.ooo>
To: incoming+gitlabhq/gitlabhq+bad_token@appmail.adventuretime.ooo
Message-ID: <CADkmRc+rNGAGGbV2iE5p918UVy4UyJqVcXRO2=otppgzduJSg@mail.gmail.com>
Subject: New Issue by email
Mime-Version: 1.0
Content-Type: text/plain;