Skip to content

Draft: Add publishing of ActivityPub activities for the releases actor

kik requested to merge gitlab-community/gitlab:activitypub-releases-activity into master

⚠️ Overarching MR. This MR will be split in several more digestable small MRs, this one is not intended to be merged. It's an opportunity to see/test the whole feature, and to run the full test suite on it.

The feature worked on here is part of the series adding support in GitLab for the ActivityPub protocol. This will allow people on the Fediverse (a decentralized social media) to subscribe to events from GitLab, for example to be notified when a project makes a new release. Ultimately, the same protocol will be used to allow to do cross-instance merge requests between several instances of GitLab.

The issue being worked on here is about allowing to subscribe to a first "actor" (that's the ActivityPub term for "resource"), which is the "releases" actor (when a project create new releases). We need to allow people from the Fediverse to subscribe to the actor (by posting a JSON payload to an endpoint, called the actor's inbox), and then we need to send the news ("activities", in ActivityPub terminology), when a new release is created.

The previous overarching MR introduced the subscription part of following releases for a project on GitLab from the Fediverse, which includes both following and unfollowing.

This overarching MR introduces sending out the activities to the third party servers when the event is triggered. Its child MRs are:

Activity

An Activity in the ActivityPub protocol is a JSON object following the structure defined in Activity Streams 2.0 and ActivityPub documents.

It means it needs:

  • the @context field to define its namespace (think: HTML doc type)
  • the id field to give the activity a unique identifier
  • the actor field to identify who is performing the activity (it's the "subject" part of the activity, usually a URL for a user profile)
  • the type field to tell what kind of activity it is (it's the "verb" part of the activity)

Most activities also have an object field, to describe on what the action is performed, which can be an other actor or an object.

In our case here, the actor is the author of the release, the type is "Create", and the object is an Application actor. That type may change later when we test integration with Mastodon, but it already has been defined in an other MR (the same activities are listed on an endpoint), so we'll keep like that for now.

Sending activities out

When we want to publish something, we send activities out by pushing them on subscriber's servers. So, unlike RSS where all those servers and periodically and constantly pulling a page to see if something changed, most of the time only to realize that nothing changed, we take the initiative of pushing when something actually happened, replacing the pull model with a push model.

This already dramatically reduce the load the feature causes, but ActivityPub goes further by adding shared inbox. In ActivityPub terminology, the inbox is the endpoint owned by a subscriber (typically a user) where we push our activity to let them know something changed. Given ActivityPub is a federated protocol, where users are registering to a server and then servers communicate between each others (like SMTP), ActivityPub takes advantage of the fact that many users may be on the same server and implement shared inbox : if 1000 users are on the same server, we send the activity only once on the shared inbox, and the third party server is responsible for dispatching it to all subscribers, which allows us to do a single request instead of 1000.

All major implementations of ActivityPub implement shared inbox, but it's optional in the specification, so we also handle subscription with no inbox: the worker will batch send activities to all shared inbox, and when there is no more, it will batch send to individual inboxes.

Adding limits to how many subscriptions can be created by an individual inbox, or even by a given server will be the subject of a further issue, which will be implemented before we remove the feature flag.

How to set up and validate locally

A Sinatra app has been created to test communication with the application, as this is a server to server feature:

  1. prepare your GitLab local installation by making the flightjs/Fligthjs project public, enabling the activity_pub and activity_pub_project feature flags and allowing requests to the local network from webhooks and integrations. This can be done in a rails console with the following:
Feature.enable(:activity_pub)
Feature.enable(:activity_pub_project)
ApplicationSetting.first.update(allow_local_requests_from_web_hooks_and_services: true)
flight = Project.find_by_name("Flight")
flight.update(visibility: "public")
  1. Install and run the sinatra app:
git clone https://gitlab.com/oelmekki/activitypub-review
cd activitypub-review
bundle
ruby server.rb
  1. Visit the app URL and click the "subscribe" button
  2. You should see the success reply from GitLab (it may takes a while on first request in dev env)
  3. You have created a subscription. To verify that, you can go check the Sinatra logs, and you should see:
  • a request to /subscriber/inbox, this is the GitLab server sending the Accept activity
  • a dump of the request object, you can verify it correctly sets the Accept http header to application/ld+json; profile="https://www.w3.org/ns/activitystreams"
  • a dump of the activity posted, as a JSON object, with type "Accept" and containing the initial Follow activity as object.

⚠️ Note: if you want to run the test several times, you need to delete the subscription between each time: ActivityPub::ReleasesSubscription.destroy_all

Now let's test emitting activities, which is the purpose of this MR:

  1. Create a new release for the Flight project in GitLab:
flight = Project.find_by_name("Flight")
user = User.first
FactoryBot.create(:release, project: flight, author: user, released_at: Time.now)
  1. In rails console, invoke the cron task:
Releases::PublishEventWorker.new.perform

⚠️ If someone knows a better way to trigger cron tasks in development environment, please let me know!

  1. Then go check the Sinatra's app log. You should see:
  • a request to /subscriber/inbox, this is the GitLab server sending us the activity
  • a dump of the request object, you can verify it correctly sets the Accept http header to application/ld+json; profile="https://www.w3.org/ns/activitystreams"
  • a dump of the activity posted, as a JSON object, with type "Create" and information about the release as object.

Related to &11247

Edited by kik

Merge request reports