spark

Process MediaWiki/Wiktionary database dumps

Graduate Tools and Models - Data Science.

Rice University, Spring 2024.

Дипломный проект с составлением датасета и его использованием для машинного обучения с целью кластеризации.

"Cloud container data analytics, statistical modeling, and machine learning on distributed databases". "A free opensource alternative to SPSS, SAS, MATLAB, PowerBI, Tableau and Alteryx". Runs on Linux, Windows, MacOS, and in the cloud via containers.

Strategic Information Management - Hadoop with Spark

This web app finds the best configuration of a Spark Application given the hardware of the cluster

Library for MQL5 (MetaTrader) with Python, Java, Apache Spark, AWS https://roffild.com/

From Data ASOS (https://mesonet.agron.iastate.edu/request/download.phtml), Analysis of aviation data to underline some patterns

Spatial join of geospatial data from Kafka streams using Apache Spark (Spark Streaming).

This project is a realtime streaming data collection system based on kafka and spark.

Spark demo project

This is my take on 'get tweeter stream, send it over to Apache Kafka as JSON objects, receive it in Apache Spark and process the stream'

This repo presents performance comparisons between a serial implementation, a MPI based and a Spark based implementation of a document clustering algorithm

This code will parse spark-defaults.conf file and make changes to Spark driver extraclasspath configuration and Spark executor extraclasspath configuration. This was written for Amazon cluster running spark.