PySpark : Combining Machine Learning & Big Data

With the ever increasing flow of data, comes the industry focus on how to use those data for driving business & insights; but what about the size of the data these days, we have to deal with ?

The more cleaner data you have, its good for training your ML ( Machine Learning ) models, but sadly neither the world feeds you clean data nor the huge amount of data is capable of fast processing using common libraries like Pandas etc.

How about using the potential of big data libraries with support in Python to deal with this huge amount of data for deriving business insights using ML techniques? But how can we amalgamate the two?

Here comes “ **PySpark : Combining Machine Learning & Big Data** “.

Usually people in the ML domain prefer using Python; so combining the potential of Big Data technologies like Spark etc to supplement ML is a matter of ease with pyspark ( A Python package to use the Spark’s capabilities ).


**This talk would revolve around** -

1) Why do we need to fuse Big Data with Machine Learning ?

2) How Spark’s architecture will help us boost our preparations for faster ML ?

3) How pyspark’s MLlib ( Machine Learning library ) helps you do ML so seamlessly ?


Resources

Ayon Roy

Ayon Roy

Data Science Intern at Lulu International Exchange

Other sessions from: Global AI Student Conference

The current state and future of AI

The current state and future of AI

In this roundtable, we get together with AI researchers and evangelists to...

Grigory Sapunov Grigory Sapunov
Mikhail Burtsev Mikhail Burtsev
Sergey Markov Sergey Markov
Dmitry Soshnikov Dmitry Soshnikov
Teaching your Models to play fair

Teaching your Models to play fair

It is very important to ensure fairness while building an AI system which...

Rishit Dagli Rishit Dagli
Introduction to Machine Learning and an overview of popular algorithms.

Introduction to Machine Learning and an overview of popular algorithms.

This session would be meant for both beginners and intermediate level stude...

Annanya Vedala Annanya Vedala
How to Build Successful Career in AI/ML

How to Build Successful Career in AI/ML

In this roundtable, we will hear different opinions on what would be the...

Shana Matthews Shana Matthews
Locksley Kolakowski Locksley Kolakowski
Syed Farhan Ahmad Syed Farhan Ahmad
Ayon Roy Ayon Roy
Annanya Vedala Annanya Vedala
Real Time Object Detection With TensorFlow

Real Time Object Detection With TensorFlow

In this session, I will discuss about my project "Sign language detection...

Nigama Vajjula Nigama Vajjula
Learning AI/ML: Is University the best place to do it?

Learning AI/ML: Is University the best place to do it?

With many teaching resources available online, including reputable Machine...

Lee Stott Lee Stott
Ajit Jaokar Ajit Jaokar
Anandha Gopalan Anandha Gopalan
Noah Gift Noah Gift
Ayse  Mutlu Ayse Mutlu