Sign in Sign up

Learning path

Big Data with Apache Spark

Why big data needs Spark, DataFrames and Spark SQL, and machine learning with MLlib, with animated films and runnable Python. Part of the Data Scientist premium track.

Start with lesson 1
  1. 1

    Big Data: Why One Computer Isn’t Enough (Hadoop, MapReduce, Spark)

    When data outgrows one machine: distributed storage (HDFS), the MapReduce idea, and why Apache Spark became the standard for big data processing. With a runnable MapReduce word count.

    3 min read
  2. 2

    PySpark DataFrames and Spark SQL

    Read big files with PySpark, filter, add columns, group, join and query with Spark SQL, understand lazy evaluation and partitions, and compare every step with pandas.

    3 min read
  3. 3

    Machine Learning at Scale with Spark MLlib

    Train models on huge datasets with Spark MLlib pipelines: VectorAssembler, StringIndexer, logistic regression and random forests, evaluation, and how it maps to scikit-learn pipelines you can run.

    3 min read