I've been using Spark for close to 2 years now but because I've always used it largely on clusters at work, I've never really had to struggle with the minutae of enabling monitoring pages like the Spark History Server UI etc.
Now that I'm exploring some advanced concepts in Spark, one of the first things I learnt was enabling the Spark History Server locally i.e on a Standalone installation.
In their own words, FFmpeg is "A complete, cross-platform solution to record, convert and stream audio and video."
I've been recording Bangpypers videos for the last few months and I haven't really had access to a proper solution to edit videos and audio that I could have got for free/ low-cost. A good friend of mine, Vinay Keerthi, (who incidentally presented the webinar under discussion) told me to chuck GUI based fronts for editing them and told me to try FFmpeg, the CLI tool directly, which these tools probably use in the background anyway.
Jupyterlab is a definite improvement on the older IPython notebook interface - both in features and in appearance. There is now even an in-built "Dark Theme" that can be enabled.
I'm using Ubuntu 16.04 on this machine, so that's what the steps of installation here will be for. But this installation shouldn't largely vary on any distro that's 14.04 and higher.
Disclaimer : Extensions in Jupyter-lab are still very much experimental. But this one seems to be working fabulously so far.
jupyterlab-git is an extension that lets you stage and commit changes to notebooks made right from within the Jupyterlab interface.
These are the commands I use a lot using Pandas -
I attended an interesting Webinar titled "A Data Science Playbook for Explainable ML/AI" conducted by Chief Data Scientist,Josh Poduska, and VP of Marketing, Jon Rooney of Domino Data. This post lays out some of the highlights (in my opinion) of the talk, peppered with my understandings and some additional points that might be of interest. I will breaking this learning-post up into two parts - the first part discussing the theoretical concepts of interpretability and Part-2 on different Open Source models that are in practice today. This is Part-1.
A guide to installing a newer version of Python is available here.
We at Bangpypers conduct a lot of workshops and some steps are common for all of them. To that end, this post is meant to serve as guide for people to install Python on Ubuntu and Windows (if required), setting up virtualenv and installing the package(s) germane to the corresponding Workshop.
The previous post was for commands Unix. This post is about the Python commands I use or Google frequently.