On using less popular R packages and data validation

Note: This post is geared towards data practitioners. If you don't fall into this group, feel free to ignore.

I have a saying about R (probably in Python, I know its in R) -- There's a package for everything. The beauty of open source languages is that anyone can write their own packages [libraries] and publish them on Github for anyone to download. The vast majority of useful R packages are collected and hosted in the CRAN package repository, which currently features over 18,000 packages.

When I say that there's a package for everything, I really mean it. There are the "standard" packages to augment base R, like the Tidyverse to help clean and structure data. But there are also more niche packages that can help you do useful things like import SAS files. There are also totally random packages, like the awesome Brooke Watson's package that can make your computer output Rapper Adlibs when your script is finished running (I think my favorite is Waka's, but the lack of Kanye in the package is borderline criminal).

This luxury comes with a price, however. Like our Middle School teachers warned us about Wikipedia,

Comments

Why isn't Robinhood letting me trade? (hint: there's probably not a conspiracy against you)

Today's been a big day in the stock market . Lots of people have lost a lot of money, and a lot of people are understandably really upset . Here's a quick breakdown of what's happened so far A subreddit called /r/wallstreetbets (visit at your own peril), which has exploded in popularity recently and has over 5 million subscribers (and counting) got really excited about three stocks: GME (Gamestop), AMC (the movie theater place), and BB (Blackberry). Gamestop was the main stock. Yes, I know all three companies are doing terribly in the real world. I won't go into why they got excited about the stocks here. They convinced a lot of other people to buy the stocks and they did well. Really well. Take a look at their Yahoo Finance pages and look at their 1 month price charts (then ignore the past two days). GME , BB , AMC Everyone got in on it, and I mean it. When a lot of people buy a single stock, the price rises. It turns out, this was hurting a lot of Hedge Funds and I...

Determining NFL Quarterback Archetypes (with stats!)

We're obsessed with grouping things together. We self-select each other into groups based on which political candidate we support, which sports team we root for, and which arbitrary country we're born in. People also spend hours on the internet arguing over "tiers", or groupings, of their favorite athletes and sports teams. For example, which NBA players are "elite" vs. "great" vs. just "good"? Did Carmelo Anthony belong on the Banana Boat ? When engaging in these arguments, we typically use statistics like points or rebounds per game to back up our points, but at the end of the day, the groups are more or less kind of arbitrary. But what if there was a way to algorithmically sort observations into groups based on shared characteristics using machine learning methods? Enter clustering , which is the methodology of grouping similar observations into groups, or "clusters", using a mathematical distance metric derived from a set ...

The Minimum Wage, the Living Wage, and the Wardrobe

The Senate is currently in intense debate regarding raising the federal minimum wage. Several potential wages have been proposed, including a $10/hour plan from Senators Romney and Cotton and a more generous $15/hour plan from the progressive Democrats. Right now the current federal minimum wage stands at $7.25 per hour, which 21 states (including my notably Blue home state of Virginia) adhere to. While the debate rages on, I wanted to take a closer look at the history of the minimum wage, the concept of a "living wage", and how these two terms invariably tie together across the United States. More importantly, at some point, there are diminishing returns and increasing costs to increasing the minimum wage. So where should we settle? The History of the Minimum Wage This isn't a history blog, so I'll be brief. The minimum wage was established under the Fair Labor Standards Act in 1938 and set at $0.25/hour, which is worth around $4.60/hour today. Since then, it has ...

Statistics with Sasa

Search This Blog