/api/v4/projects/{}/pipelines/{}/jobs is not accessible by CI_JOB_TOKEN though it's the same project

Summary

(My company is a premium customer, running a self-hosted instance)

I'm trying to implement a gitlab-runner executor on AWS Lambda, it works quite fine, except for artifacts I try to emulate the artifacts retrieving mechanism by doing

1. The runner transmit to AWS Lambda the CI_JOB_TOKEN
2. My lambda calls /api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs with the token CI_JOB_TOKEN

but I got a 401, while with a personnal token, it works

Steps to reproduce

In a CI job run the following script

 curl -H "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab./api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs?scope%5B%5D=success"

Example Project

https://gitlab.com/allan-simon/reproduce_api_bug/-/jobs/359386402#L20

What is the current bug behavior?

/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs can not be called by a runner executing a job in that very same project and pipeline

What is the expected correct behavior?

this api should be callable with CI_JOB_TOKEN corresponding to that project and pipeline

Output of checks

This bug happens on GitLab.com (and managed instance)