...
 
Commits (7)
......@@ -4,18 +4,33 @@
Before you can get started developing ARSnova you need to make sure the following software is installed to build ARSnova Backend:
* Java 8 JDK
* OpenJDK 8 or 11 Development Kit
* Apache Maven 3.x
And additionally if you want to run ARSnova locally:
* Apache CouchDB 1.x
* Python 2.7
* [ARSnova Setup Tool](https://github.com/thm-projects/arsnova-setuptool)
* Apache CouchDB 2.x (see [Installation Guide](installation.md#couchdb))
Next, you need to setup an ARSnova configuration file.
Create a copy of [arsnova.properties.example](../../main/resources/arsnova.properties.example) at `/etc/arsnova/arsnova.properties`.
Afterwards, initialize the database by running the `tool.py` python script from the Setup Tool.
Create a new file [application.yml] at a location of your choosing outside of the repository:
```yaml
arsnova:
system:
root-url: http://localhost:8080
api:
expose-error-messages: true
couchdb:
host: localhost
db-name: arsnova3
username: <couchdb admin user>
password: <couchdb admin password>
security:
jwt:
secret: <random string for encryption/signing>
```
Have a look at [defaults.yml](../../main/resources/config/defaults.yml) for an overview of all available configuration settings and their defaults.
## Building
......@@ -33,12 +48,15 @@ You can create a web archive (`.war` file) by running a single command:
ARSnova builds are setup up to automatically download the Java Servlet container Jetty for development.
Run the following command to download the dependencies, and startup the backend with Jetty:
$ mvn jetty:run
$ mvn jetty:run -D arsnova.config-dir=</path/to/config>
After a few seconds the ARSnova API will be accessible at <http://localhost:8080/>.
You can adjust the amount of debug logging by changing the log levels in [log4j-dev.properties](../../main/resources/log4j-dev.properties).
Additionally, you can enable exception messages in API responses by setting the boolean property `api.expose-error-messages` in `arsnova.properties`.
You can customize the logging behavior for the development environment by appending the following parameters:
* -D arsnova.log.level=TRACE
* -D arsnova.log.level.spring=DEBUG
* -D arsnova.log.exeptions=5
## Continuous Integration
......@@ -46,7 +64,7 @@ Additionally, you can enable exception messages in API responses by setting the
Our code repositories are located on a [GitLab server](https://git.thm.de/arsnova) for internal development.
They are automatically mirrored to [GitHub](https://github.com/thm-projects) on code changes.
Apart from mirroring GitLab CI triggers various jobs to:
Apart from mirroring GitLab CI runs various jobs to:
* check the code quality (static code analysis with SonarQube)
* build a web archive
......@@ -65,5 +83,8 @@ The current build status for the master branch:
## Further Documentation
* [Roadmap](development/roadmap.md)
* [Domain](development/domain.md)
* [Layers](development/layers.md)
* [Caching](development/caching.md)
* [Event System](development/event-system.md)
* [API](development/api.md)
# API
## Authentication
The ARSnova API uses _JSON Web Tokens_ to authenticate users statelessly.
To receive a JWT initially, a client has to login using an endpoint specific for the authentication provider.
As response a JSON object containing the JWT is sent:
```json
{
"userID": "fe5d046b30c64e3e931094865d3415c4",
"loginId": "arsnova-user@example.com",
"authProvider": "ARSNOVA",
"token": "eyJ0eXAiOi...eLvinH9rZ0"
}
```
To get authorization to API endpoints the JWT is sent as part of HTTP Basic Authentication with `Bearer` scheme as described in
[RFC 6750](https://tools.ietf.org/html/rfc6750).
The `Authorization` header is set for all further requests:
```
Authorization: Bearer eyJ0eXAiOi...eLvinH9rZ0
```
## REST Endpoints
## Query Endpoint
## Management Endpoints
# Caching
Please read about Spring Framework's [Cache Abstraction](http://docs.spring.io/spring/docs/current/spring-framework-reference/html/cache.html) first.
Please read about Spring Framework's [Cache Abstraction](https://docs.spring.io/spring/docs/5.1.x/spring-framework-reference/integration.html#cache) first.
## What should get cached?
The short answer: All data that is written once and read multiple times. In ARSnova, there is an inherent `1:n` relationship between teachers and students. This makes everything the teacher creates a candidate for caching. But there are more opportunities like students' answers which are mostly written once and cannot be changed afterwards. Be aware though that in this case, once a new answer comes in, the cache has to be invalidated. With many students answering questions at the same time, the effects of caching go away since the cache is invalidated all the time.
The short answer: All data that is rarely written but frequently read.
Caching is already handled for all entities as long as they are accessed through the `DefaultEntityServiceImpl`.
Additional, specialized caching can be helpful for aliases (e.g. mapping a room's `shortId` the `id`) or find query results.
While caching provides an opportunity to greatly speed up the execution of various requests, it does come with a price: You have to think of all cases were the cached data might become stale.
While caching provides an opportunity to greatly speed up the execution of various requests, it does come with a price:
You have to think of all cases were the cached data might become stale.
## How to design your objects
## Generating cache keys
Caching should only be used with domain objects where the `hashCode` and `equals` methods are provided. This makes it easy to update or delete cache entries. As you recall from the documentation, cache keys are based on a method's parameters. If you use base objects like `String` or `Integer`, you will have to manually provide a key through the Spring Expression Language (SpEL). As you can see from the following example, such keys can become quite complicated:
By default, the cache key is generated from all method parameters.
If some of the parameters should not affect cache handling, the key parameter of the caching annotation has to be set.
For reference types which are used as part of the key you have to make sure that `hashCode` and `equals` methods are overriden.
Be aware though that you need to carefully choose the fields which should be part of the `equals`/`hashCode`:
In case of CouchDB, for example, it is not a good idea to use a document's `rev` field.
Every time a document is updated, it gets a new `rev` which will make it _unequal_ to all its previous versions,
making cache updates using `@CachePut` impossible.
```java
@Cacheable(value = "notverycacheable", key = "#p0.concat('-').concat(#p1).concat('-').concat(#p2)")
public ResultObject notVeryCacheable(String roomId, String questionVariant, String subject) { ... }
```
Therefore, you should always work with domain objects like `Room`, `Content`, or even your own, newly defined objects:
## Event-based cache updates
```java
@Cacheable("verycacheable")
public ResultObject veryCacheable(Room room) { ... }
```
[ARSnova's event system](event-system.md) provides a useful way for fine-grained cache updates because the events contain all relevant domain objects.
If you need to update a cache based on one of ARSnova's events, you can combine `@EventListener` with one of the caching annotations.
Be aware though that you need to carefully choose the fields which should be part of the `equals`/`hashCode`: In case of CouchDB, for example, it is not a good idea to use a document's `rev` field. Every time a document is updated, it gets a new `rev` which will make it _unequal_ to all its previous versions, making cache updates using `@CachePut` impossible.
[ARSnova's event system](event-system.md) provides a useful way for fine-grained cache updates because the events contain all relevant domain objects. If you need to clear or update a cache based on one of ARSnova's events, you can use the `CacheBuster` class to add your annotations.
## Issues
Caching requires the use of Spring Proxies. This means that methods invoked using `this` ignore all caching annotations! They only work across object boundaries because Spring is only able to intercept calls if they are going through a Spring Proxy. This could only be solved using AOP, but we have no intention to support this in the near future.
There is one exception: Since the `databaseDao` bean needs to call several methods on the same object, we implemented a workaround that allows access to the bean's proxy. When `getDatabaseDao()` is called within the bean, its proxy is returned that should be used instead of `this`.
One last word of caution: Your code should not rely on the cache's existence, and you should keep expensive calls to a minimum: Do not hit the database multiple times even though you think further calls are served by the cache.
## List of cache entries and associated keys
## List of caches
Here is a list of all caches, their keys, and a short description.
_Note_: With the introduction of the generic `entity` cache many of the other entity related caches became obsolete and will be removed in the future.
Cache name | Key | Description
-----------|-----|------------
`entity` | entity type + `-` + id | Contains all entity objects handled by `DefaultEntityServiceImpl`.
`contentlists`| database id of room | Contains all contents for the specified room irrespective of their variant.
`lecturecontentlists` | database id of room | Contains all "lecture" variant contents for the specified room.
`preparationcontentlists` | database id of room | Contains all "preparation" variant contents for the specified room.
......
# Domain
## Model
```plantuml
' You can copy and paste this code section to http://www.plantuml.com/plantuml/ to render the diagram.
@startuml
Room *-- ContentGroup
ContentGroup *-- Content
Content <|-- ChoiceQuestionContent
ChoiceQuestionContent *-- AnswerOption
Content *- TextAnswer
Answer <|-- ChoiceAnswer
Answer <|-- TextAnswer
ChoiceAnswer --* ChoiceQuestionContent
ChoiceAnswer -- AnswerOption : selects >
Room *-- Comment
Room "0..1" *-- Motd
Room - "1" User : < owns
Room - User : < moderates
Comment -- "1" User : < creates
Answer -- "1" User : < creates
hide circle
hide empty fields
hide empty methods
skinparam monochrome true
skinparam shadowing false
@enduml
```
Please note: This is a domain diagram and its purpose is to give a quick overview of the relations between entity classes.
The concrete implementation differs and most relations use indirect references via IDs or indexes.
## Entities
* `Room`s are key to access and organize data for a group of participants.
Most entities are directly or indirectly related to a room.
While the denotation "room" is consistently used for code,
"session" is used in some legacy APIs and might also be used in clients for GUI labels.
* `Comment`s store short texts written by participants of a room.
* `ContentGroup`s are used to classify a group of contents (e.g. by topic or date) for a room.
* `Content`s are used to provide text content for participants in a specific format.
In case of `ChoiceQuestionContent` content is in the form of a question with predefined `AnswerOption`s.
* `Answer`s store a text (`TextAnswer`) or selected options (`ChoiceAnswer`) as a reaction from a participant to a content.
* `Motd`s (Messages of the Day) store informational texts for a specific room or the whole ARSnova instance.
# Event System
ARSnova's event system allows objects to act upon changes to any of ARSnova's data structures. For example, if a new question is created, a `NewQuestionEvent` is sent out and any object that is interested can receive the question itself along with meta data such as the relevant `Session`. Among other use cases, the event is used to notify clients via Web Socket.
ARSnova's event system allows components to act upon state changes without requiring complex bean dependencies.
Events offer a way to dynamically update data without the need to poll the database for changes.
For instance, if clients are interested in the number of currently available contents,
the respective component could initialize the number using the database, while keeping it up to date using events.
Events offer a way to dynamically update data without the need to poll the database for changes. For instance, if clients are interested in the number of currently available questions, the respective Bean could initialize the number using the database, while keeping it up to date using events.
Events are published before and after all CRUD operations of entities which are handled by `DefaultEntityServiceImpl`.
Those `CrudEvent`s contain the entity and, based on the operation specific implementation, additional details like the changes applied to an entity.
Before you start working with events you should first read
[Spring's documentation](https://docs.spring.io/spring-framework/docs/5.1.x/spring-framework-reference/core.html#context-functionality-events)
about them.
## How to send events?
A class is able to send events by implementing the `ApplicationEventPublisherAware` interface. It consists of one method to inject an `ApplicationEventPublisher`, which can then be used to send the events like so:
```java
publisher.publishEvent(theEvent);
```
where `theEvent` is an object of type `ApplicationEvent`. For ARSnova, the base class `ArsnovaEvent` should be used instead. All of ARSnova's internal events are subtypes of `ArsnovaEvent`.
_Note_: Events are sent and received on the same thread, i.e., it is a synchronous operation.
## Receiving events
Events can be received by any component by adding a listener.
A listener is a simple method which is annotated with `@EventListener` and has a single parameter, the event.
The following code snipped shows how an EventListener on the `BeforeDeletionEvent` of `UserProfile`s is registered:
## How to receive events?
Events are received by implementing the `ApplicationListener<ArsnovaEvent>` interface. The associated method gets passed in a `ArsnovaEvent`, which is the base class of all of ARSnova's events. However, this type itself is not very useful. The real type can be revealed using double dispatch, which is the basis of the Visitor pattern. Therefore, the event should be forwarded to a class that implements the `ArsnovaEvent` interface. This could be the same class that received the event.
```java
@EventListener
public void handleUserDeletion(final BeforeDeletionEvent<UserProfile> event) {
final Iterable<Room> rooms = getRoomsForUserId(event.getEntity().getId());
delete(rooms);
}
```
_Note_: If the class implementing the Visitor needs to have some of Spring's annotations on the event methods, like, for example, to cache some values using `@Cacheable`, the Listener and the Visitor must be different objects.
## Sending events
## How to create custom events?
Events are published through the `ApplicationEventPublisher` bean.
This bean can be injected to any component by either implementing Spring's `ApplicationEventPublisherAware` interface or using constructor injection.
Once the component is aware of the publisher, events be sent via its `publishEvent` method.
To be eligible as a event type, a class has to extend Spring's `ApplicationEvent`.
Subclass either `ArsnovaEvent` or `RoomEvent`. The former is for generic events that are not tied to a specific room, while the latter is for cases where the event only makes sense in the context of a room.
_Note_: Events are sent and received on the same thread, i.e., it is a synchronous operation.
# Layers
## Constraints
ARSnova's backend architecture can be separated roughly into three layers which build on one another.
Ordered from highest to lowest these layers are:
1. Controller layer
2. Service layer
3. Persistence layer
When components of another layer are accessed the following constraints have to be satisfied:
* Components of a lower layer **may not** access components of a higher layer.
* Components **may not** directly access components below the next lower layer.
Failure to satisfy these contraints could lead to unintended side-effects because important system aspects, e.g. security and caching, are bypassed.
## Entity system
```plantuml
' You can copy and paste this code section to http://www.plantuml.com/plantuml/ to render the diagram.
package org.springframework.data.repository {
interface Repository<T, ID>
interface CrudRepository<T, ID> {
findById(id: ID): Optional<T>
findAll(ids: Iterable<ID>): Iterable<T>
save(entity: T)
saveAll(entity: Iterable<T>)
delete(entity: T)
deleteAll(entities: Iterable<T>)
...()
}
}
package com.fasterxml.jackson.databind {
class ObjectMapper
}
package model {
abstract class Entity {
id: String
rev: String
creationTimestamp: Date
updateTimestamp: Date
-internal: boolean
}
Entity <|- ConcreteEntity
}
package controller {
abstract class AbstractEntityController<E: Entity> {
get(id: String): E
getMultiple(ids: Collection<String>): Iterable<E>
put(entity: E, response: HttpServletResponse): E
post(entity: E, response: HttpServletResponse): E
patch(...): E
delete(id: String)
find(findQuery: FindQuery<E>): Iterable<E>
...()
}
AbstractEntityController <|-- ConcreteEntityController : <<bind>>\n<E -> ConcreteEntity>
}
package service {
interface EntityService<E: Entity> {
get(id: String): E
get(ids: Iterable<String>): Iterable<E>
create(entity: E): E
update(entity: E): E
update(oldEntity: E, newEntity: E): E
patch(entity: E, changes: Map<String, Object>): E
patch(entities: Iterable<E>, changes: Map<String, Object>): Iterable<E>
delete(entity: E)
delete(entities: Iterable<E>
...()
}
class DefaultEntityServiceImpl<E: Entity> {
objectMapper: ObjectMapper
}
EntityService <|.. DefaultEntityServiceImpl
DefaultEntityServiceImpl <|-- ConcreteEntityService : <<bind>>\n<E -> ConcreteEntity>
}
package persistence {
Repository <|-- CrudRepository
CrudRepository <|.. ConcreteRepository : <<bind>>\n<T -> ConcreteEntity\nID -> String>
}
ConcreteEntityController o- ConcreteEntityService
ConcreteEntityService o- ConcreteRepository
hide circle
'hide empty fields
'hide empty methods
skinparam monochrome true
skinparam shadowing false
@enduml
```
This diagram shows the layers and classes involved in handling entities. Its intention is to give a brief overview, so some elements have been left out for simplicity.
......@@ -115,9 +115,9 @@ A new streamlined REST API is developed while the legacy API is still supported
* Implement a `/<entity type>/find` endpoint
* Provide statistics with non-aggregated combinations of answer choices
* Implement customizable content groups
* They replace predefined content variants
* Relations are stored as part of a room
* They support auto sort (v2 behavior) and user defined ordering
* They replace predefined content variants
* Relations are stored as part of a room
* They support auto sort (v2 behavior) and user defined ordering
* Implement API live migration layer v2 <-> v3
* v2 controllers convert between v2 and v3 entity types
* Internally (service and persistence layer) only the v3 entity type is used