Why it matters: With datasets becoming larger on average, understanding how to effectively work with data in memory boosts your productivity and flexibility as a data professional. Understanding how to process these data in pandas will allow you to manage data at scale with ease.
I've spent over 4 years in an academic, start-up like environment providing data science solutions across our laboratory, developing entire image processing pipelines for large, complex image datasets. I tackle all our data science problems from preprocessing to data visualization, and statistical conclusions. I also apply bioinformatics workflows with machine learning techniques in HPC environments to process large amounts of data in parallel for projects in cheminformatics, genomics, transcriptomics, and proteomics.