[deleted]

Comments

You must log in or register to comment.

timelyparadox t1_j5iu8hi wrote on January 23, 2023 at 8:31 AM

Are you using Spark?

[deleted] OP t1_j5ivup4 wrote on January 23, 2023 at 8:53 AM

No, I’m not. Can you share any reference material

timelyparadox t1_j5iw0j4 wrote on January 23, 2023 at 8:55 AM

Medium article:

https://medium.com/analytics-vidhya/how-to-pre-process-large-datasets-for-machine-learning-using-spark-19500155b521

Not perfect example but might be helpfull to lead into right direction.

developers_hutt t1_j5kwuq5 wrote on January 23, 2023 at 7:06 PM

Use your PC
You can use libraries like RAY or PySpark.

[deleted] OP t1_j5kwzqk wrote on January 23, 2023 at 7:07 PM

I think I just need to start working with spark

developers_hutt t1_j5kxaun wrote on January 23, 2023 at 7:09 PM

Are you using Python? Then Pyspark will be easy for you.

[deleted] OP t1_j5kx8iy wrote on January 23, 2023 at 7:08 PM

[deleted]

[deleted] OP t1_j5kxa5u wrote on January 23, 2023 at 7:09 PM

Yes, python