8000 GitHub - LucaCanali/Miscellaneous: Includes notes on using Apache Spark, with drill down on Spark for Physics, how to run TPCDS on PySpark, how to create histograms with Spark. Also tools for stress testing and measuring CPUs's performance. Jupyter notebooks examples for using various DB systems.
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Includes notes on using Apache Spark, with drill down on Spark for Physics, how to run TPCDS on PySpark, how to create histograms with Spark. Also tools for stress testing and measuring CPUs's performance. Jupyter notebooks examples for using various DB systems.

License

Notifications You must be signed in to change notification settings

LucaCanali/Miscellaneous

Repository files navigation

Miscellaneous Projects, Tools, and Scripts

DOI
Contact: Luca.Canali@cern.ch

Performance Engineering and Apache Spark

Folder Description
Spark Dashboard A tool for Apache monitoring, use to build a performance dashboard and troubleshoot Spark jobs.
Spark Notes Miscellaneous tips and code snippets about Apache Spark.
Spark for Physics Examples, with code and data of using Apache Spark for High Energy Physics data analysis.
Performance Testing Includes:
- TPCDS-PySpark, run TPCDS bemchmark at scale with PySpark and collect execution metrics
- Load testing tools for CPU benchmarking, in Python and Rust
- Notes on how to use various tools for performance investigations

Data Engineering and Data Science

Folder Description
Kepler Analysis A curated collection of interactive notebooks for executing Kepler's orbital analysis on Mars.
Deep Learning Notes Notes and examples on Deep Learning tools and related data pipelines.
Pyspark_SQL_Magic_Jupyter How to write Jupyter SQL magic functions for PySpark and Spark SQL.
Trino and Presto on Jupyter Example of using Trino or Presto on a Jupyter notebook.
PostgreSQL and YugabyteDB on Jupyter Example of using PostgreSQL or YugabyteDB on a Jupyter notebook.
Oracle_Jupyter Examples of how to query Oracle using Jupyter/IPython notebooks.
Impala_SQL_Jupyter Examples of how to run SQL on Apache Impala using Jupyter/IPython notebooks.
SQL_color_Mandelbrot How to use SQL to compute and display the Mandelbrot set with colors. Examples for Oracle and PostgreSQL.
PLSQL_Neural_Network An example of neural network inference using Oracle RDBMS and PL/SQL.

About

Includes notes on using Apache Spark, with drill down on Spark for Physics, how to run TPCDS on PySpark, how to create histograms with Spark. Also tools for stress testing and measuring CPUs's performance. Jupyter notebooks examples for using various DB systems.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  
0