Add check to allow only events which are less than 24 hours old in devkit
What does this MR do and why?
This MR updates custom enricher to discard events which are 24 hours old.
custom enricher MR - custom-enrichers!4 (merged) Relates to - gitlab-org/gitlab#434471
How to set up and validate locally
- To test if changes are working fine locally, instead of 24 hours change the difference to 10s. Below is the js code that does that.
function process(event) {
if (!event.getDvce_sent_tstamp()) {
throw "dvce_sent_tstamp is not defined";
}
if (!event.getDvce_created_tstamp()) {
throw "dvce_created_tstamp is not defined";
}
const dvceSentTstamp = new Date(event.getDvce_sent_tstamp());
const dvceCreatedTstamp = new Date(event.getDvce_created_tstamp());
const timeDifference = dvceSentTstamp.getTime() - dvceCreatedTstamp.getTime();
const hoursDifference = timeDifference / (1000 * 60 * 60);
if (hoursDifference > 0.002) {
throw `The time difference between dvce_sent_tstamp (${dvceSentTstamp.toISOString()}) and dvce_created_tstamp (${dvceCreatedTstamp.toISOString()}) is greater than 24 hours`;
}
}
Base64encoded version -
ZnVuY3Rpb24gcHJvY2VzcyhldmVudCkgewogIGlmICghZXZlbnQuZ2V0RHZjZV9zZW50X3RzdGFtcCgpKSB7CiAgICB0aHJvdyAiZHZjZV9zZW50X3RzdGFtcCBpcyBub3QgZGVmaW5lZCI7CiAgfQoKICBpZiAoIWV2ZW50LmdldER2Y2VfY3JlYXRlZF90c3RhbXAoKSkgewogICAgdGhyb3cgImR2Y2VfY3JlYXRlZF90c3RhbXAgaXMgbm90IGRlZmluZWQiOwogIH0KCiAgY29uc3QgZHZjZVNlbnRUc3RhbXAgPSBuZXcgRGF0ZShldmVudC5nZXREdmNlX3NlbnRfdHN0YW1wKCkpOwogIGNvbnN0IGR2Y2VDcmVhdGVkVHN0YW1wID0gbmV3IERhdGUoZXZlbnQuZ2V0RHZjZV9jcmVhdGVkX3RzdGFtcCgpKTsKICBjb25zdCB0aW1lRGlmZmVyZW5jZSA9IGR2Y2VTZW50VHN0YW1wLmdldFRpbWUoKSAtIGR2Y2VDcmVhdGVkVHN0YW1wLmdldFRpbWUoKTsKICBjb25zdCBob3Vyc0RpZmZlcmVuY2UgPSB0aW1lRGlmZmVyZW5jZSAvICgxMDAwICogNjAgKiA2MCk7CgogIGlmIChob3Vyc0RpZmZlcmVuY2UgPiAwLjAwMikgewogICAgdGhyb3cgYFRoZSB0aW1lIGRpZmZlcmVuY2UgYmV0d2VlbiBkdmNlX3NlbnRfdHN0YW1wICgke2R2Y2VTZW50VHN0YW1wLnRvSVNPU3RyaW5nKCl9KSBhbmQgZHZjZV9jcmVhdGVkX3RzdGFtcCAoJHtkdmNlQ3JlYXRlZFRzdGFtcC50b0lTT1N0cmluZygpfSkgaXMgZ3JlYXRlciB0aGFuIDI0IGhvdXJzYDsKICB9Cn0=
- Copy base64 encoded version and put it instead of this MR's change.
- Send events.
- Change
dtm
field manually so that it is more than10s
than current time. - Verify that these events went to snowplow_bad_events table with error message.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Ankit Panchal