Trends in Atmospheric NO and NO2 Concentrations

As part of my Data Analysis in R course on Udacity, I’m publishing the results of an EDA I did on atmospheric nitric oxide and nitrogen dioxide concentrations somewhere in Cambrige (UK).

You can find the datasets here.

I plotted the levels of NO in the atmosphere over the span of several days, and noticed that it tends to have a daily cycle. The levels probably go down during the night, and go back up when it warms up during the day :

Rplot02

Viewed on a larger scale, the mean levels of NO seem to have no noticeable trend :

Rplot01

 

Plotting the levels of NO vs the levels of NO2, a noticable positive correlation emerges:

Rplot

Taking the Pearson’s product-moment correlation of NO and NO2 concentrations reveals a value of 0.6968, which supports the observation.

Here’s the R code that produced these plots, for those of you that are interested:


#http://www.airqualityengland.co.uk/local-authority/data?la_id=51

library(ggplot2)
library(dplyr)
library(grid)
library(gridExtra)

ds1 <- read.csv("2014-05-07-141107012512.csv")
ds2 <- read.csv("2014-08-05-141107012512.csv")
ds3 <- read.csv("2014-11-04-141107012512.csv")
dataset <- rbind.data.frame(ds1, ds2, ds3)


dataset$timestamp <- as.numeric(strptime(paste(dataset$End.Date,dataset$End.Time), format = "%d/%m/%Y %H:00:00"))
dataset <- dataset[!is.na(dataset$timestamp ), ]

dataset$hour <- dataset$timestamp / 3600
dataset$hour <- dataset$hour - min(dataset$hour)

dataset$day <- round(dataset$timestamp / 86400)
dataset$day <- dataset$day - min(dataset$day)

sp1 <- ggplot(aes(x = hour, y = NO), data = cleanData) +
 ylim(c(0, 150)) +
 geom_line(color = "#334455") + 
 scale_x_continuous(breaks = seq(0, 200, 24), limits = c(0, 200)) +
 labs(x = "Hour Since Start", y = "Nitric Oxide Concentration")

sp2 <- ggplot(aes(x = hour, y = NO), data = cleanData) +
 ylim(c(0, 150)) +
 geom_line(color = "#334455") + 
 scale_x_continuous(breaks = seq(0, 500, 24), limits = c(0, 500)) +
 labs(x = "Hour Since Start", y = "Nitric Oxide Concentration")

grid.arrange(sp1, sp2)



dataset.by_day <- dataset %>%
 group_by(day) %>%
 summarise(mean_NO = mean(NO))

sp1 <- ggplot(aes(x = day, y = mean_NO), data = dataset.by_day) +
 ylim(c(0, 100)) +
 geom_line(color = "#334455") +
 scale_x_continuous(breaks = seq(0, 200, 7), limits = c(0, 50)) +
 labs(x = "Days Since Start", y = "Mean NO Concentration")

sp2 <- ggplot(aes(x = day, y = mean_NO), data = dataset.by_day) +
 ylim(c(0, 100)) +
 geom_line(color = "#334455") +
 scale_x_continuous(breaks = seq(0, 200, 7), limits = c(0, 200)) +
 labs(x = "Days Since Start", y = "Mean NO Concentration")

grid.arrange(sp1, sp2)




sp1 <- ggplot(aes(x = NO, y = NO2), data = dataset) + 
 xlim(c(0, 150)) + 
 ylim(c(0, 100)) +
 geom_point(alpha = 1/5, position = position_jitter(width = 0.8, height = 0.8)) +
 geom_smooth() +
 labs(x = "Nitric oxide concentration", y = "Nitrogen dioxide concentration")

grid.arrange(sp1)

with(dataset, cor.test(x = NO, y = NO2)) # 0.6968017
1 Comment

Fuel Consumption Ratings for Canadian Vehicules in 2014

I’m currently following a course called Data Analysis in R on Udacity. Part of the course involves loading up a dataset found online, doing some exploratory data analysis, and publishing my results.

I used the dataset called “2014 – Fuel Consumption Ratings” available on this page:

http://data.gc.ca/data/en/dataset/98f1a129-f628-4ce4-b24d-6f16bf24dd64

As per the description: “A yearly data set of all passenger vehicles sold in Canada based on their fuel-consumption ratings, estimated carbon-dioxide emissions and annual fuel costs.”

I use this command to load the dataset:

fuel <- read.csv("http://www.nrcan.gc.ca/sites/www.nrcan.gc.ca/files/oee/files/excel/MY2014%20Fuel%20Consumption%20Ratings.csv");

Here’s a few commands to clean up the data frame:

names(fuel) <- c("Model.Year", "Manufacturer", "Model", "Vehicle.Class", "Engine.Size.L", "Cylinders", "Transmission", "Fuel.Type", "Fuel.Consumption.City.L.100km", "Fuel.Consumption.Hwy.L.100km", "Fuel.Consumption.City.Mpg", "Fuel.Consumption.Hwy.Mpg", "Fuel.Ly", "Co2.Emissions.g.km");
fuel <- fuel[2:1068, ];

Let’s load up the ol’ ggplot, as well as gridExtra for added flavour:

library(ggplot2);
library(grid);
library(gridExtra);
theme_set(theme_gray(base_size = 6));

First let’s coerce some rows from factors to numeric, to make plotting easier:

fuel[, "Fuel.Ly"]                         <- as.numeric(as.character(fuel[, "Fuel.Ly"]));
fuel[, "Engine.Size.L"]                   <- as.numeric(as.character(fuel[, "Engine.Size.L"]));
fuel[, "Co2.Emissions.g.km"]              <- as.numeric(as.character(fuel[, "Co2.Emissions.g.km"]));
fuel[, "Fuel.Consumption.City.L.100km"]   <- as.numeric(as.character(fuel[, "Fuel.Consumption.City.L.100km"]));
fuel[, "Fuel.Consumption.Hwy.L.100km"]    <- as.numeric(as.character(fuel[, "Fuel.Consumption.Hwy.L.100km"]));
fuel[, "Fuel.Consumption.City.Mpg"]       <- as.numeric(as.character(fuel[, "Fuel.Consumption.City.Mpg"]));
fuel[, "Fuel.Consumption.Hwy.Mpg"]        <- as.numeric(as.character(fuel[, "Fuel.Consumption.Hwy.Mpg"]));

Let’s plot a few variables to get a feel for the data:

fuelLY <- qplot(
  data = fuel, 
  x = Fuel.Ly, 
  binwidth = 100,
  xlab = "Fuel (L/year)",
  ylab = "Count"
);

co2emiss <- qplot(
  data = fuel, 
  x = Co2.Emissions.g.km, 
  binwidth = 8,
  xlab = "CO2 Emissions (g/km)",
  ylab = "Count"
);

fuelconscity <- qplot(
  data = fuel, 
  x = Fuel.Consumption.City.L.100km, 
  binwidth = 0.5,
  xlab = "Fuel Consumption in City (L/100km)",
  ylab = "Count"
);

fuelconshwy <- qplot(
  data = fuel, 
  x = Fuel.Consumption.Hwy.L.100km, 
  binwidth = 0.3,
  xlab = "Fuel Consumption on Highway (L/100km)",
  ylab = "Count"
);

grid.arrange(fuelLY, co2emiss, fuelconscity, fuelconshwy, ncol = 2);

Rplot

Looks like these variables (fuel per year, co2 emissions, fuel consumption per km) are all roughly normally distributed, and the all have a slight skew towards the left. This is a very interesting pattern. It could represent the car maker’s recent effort to make models “greener”, pushing the overall fuel efficiency up.

Let’s look at the media fuel consumption in the city broken down by manufacturer:

fuel$Manufacturer <- factor(fuel$Manufacturer); #To reset the factors

firstPlot <- qplot(
    data = fuel, 
    x = Manufacturer,
    y = Fuel.Consumption.Hwy.L.100km, 
    binwidth = 0.3,
    xlab = "Manufacturer",
    ylab = "Fuel Consumption (L/100km)",
    geom= "boxplot"
) + coord_cartesian(xlim = c(0, 16.5));

secondPlot <- qplot(
    data = fuel, 
    x = Manufacturer,
    y = Fuel.Consumption.Hwy.L.100km, 
    binwidth = 0.3,
    xlab = "Manufacturer",
    ylab = "Fuel Consumption (L/100km)",
    geom= "boxplot"
) + coord_cartesian(xlim = c(16.5, 30.5));

thirdPlot <- qplot(
    data = fuel, 
    x = Manufacturer,
    y = Fuel.Consumption.Hwy.L.100km, 
    binwidth = 0.3,
    xlab = "Manufacturer",
    ylab = "Fuel Consumption (L/100km)",
    geom= "boxplot"
) + coord_cartesian(xlim = c(30.5, 40));

grid.arrange(firstPlot, secondPlot, thirdPlot, nrow = 3);

ok1 ok2 ok3

 

(I apologize for the small images. Please click on them to see higher resolution…)

There’s many manufacturers compared to the number of 2014 models they each have, making the data a bit hard to read. Of the bigger manufacturers, Mini, Mazda and Honda have the most models with low fuel consumption, with a median near 6.

So this concludes my exploratory data analysis. Hope you enjoyed it.

Leave a comment

Starting with jMonkeyEngine – 3D Game Development Terminology

jMonkeyEngine has always caught me eye as one of the most complete and community supported 3D game engines. My Java is beginning to be very rusty, so exploring this framework works to my advantage in many ways.

As outlined in the “Community Hub“, before undertaking the beginner tutorials, it’s important to understand basic 3D game development terminology (ex. mesh, shading, specular map). They actually provide such a documentation, which is wonderfully resumed for those of us who want to get down into the action, and not spend hours reading. The scene graph is among the most important concepts, on which they have an article dedicated to.

 

Leave a comment

Redis as Session Storage – Comparison to Memcached

Yesterday I was investigating Redis as a potential key-value storage. In particular, it’s potential for session storage. I thought I should share some of my findings here.

An application I’m currently helping to maintain uses Memcached to store session data. This is bad for at least 2 reasons:

  1. Memcached data is not persisted to disk, leading to potential of closing sessions if the server is having issues,
  2. Expired keys aren’t removed automatically; they only get deleted on the next retrieval, meaning that most of the time, sessions never get deleted, because the user stops responding, and their sessions remain stale until the next instance restart

Redis was seen as a good alternative. It supports key expiration that automatically deletes keys. It also saves data to disk, preventing data loss.

 

Leave a comment

My Totally Realistic Blog Name Generator

Here it is:

Click Here to Refresh

Blog Name Generator

1 Comment

How to Consistently Score 100% Grades in Calculus (and Maths) – On Practicing

In this article, I’m going to help you achieve perfect grades in Calculus class. If you have great determination, 100% is reachable, and you won’t have to sell your soul to get there.

Today, I’m going to focus on the art of practising intelligently.

Practice Often

I consider this the most important piece of advice, yet also the most obvious, that I can offer you. You aren’t born with great mathematics abilities, you need to develop them! Scientists are learning more and more that the brain is very adaptable, and never stops changing. By practising calculus often,  you’re creating new connections, reinforcing good habits, and speeding up the logical thinking necessary to solve problems efficiently.

Like any habit, the more often you exercise it, the more your brain makes it natural. This means practising at different times of day, and practising on a consistent basis.

Practice Thoroughly

Textbooks contain thousands of problems for a very good reason; because you’ll need to drill the fundamentals of calculus into your head. It isn’t sufficient to understand why a certain theorem or law exists; you need to use it over and over again until you know it and understand it personally & intuitively.

In calculus textbooks, you’ll often have different types of problems for each chapter:

Problems that help you understand a theorem or technique

These are very important to do. They’ll let you know if you understand the basic premise of the chapter.

Problems that give you the opportunity to put it into practice

This is the meat of the chapter. It’s the drill that will commit the technique to your mind. Do all of these.

Problems that apply the technique to real-world problems

I personally don’t find these very important. They might be helpful if you’re having a hard time finding the practical application of a technique. It will probably be more helpful to kinesthetic learners.

Challenging or integrative problems

These are very useful as review problems. They will let you know if you’re ready for an exam, or to move on to the next chapter.

How to Practice

I follow these steps when going through the problems section of my textbook:

  1. Start from the beginning
  2. Do each problem one after another, checking to see if you have the right answer after each
  3. If you make a mistake, redo the problem the right way. Otherwise, you’re not learning from your mistakes.
  4. Don’t move on to the next problem until you 100% understand your mistakes in the last one
  5. You may skip a question only if the answer comes very naturally to you

Don’t get frustrated from your mistakes. You have to understand that making mistakes is one of the most important parts of learning. In other words, you must make mistakes. Everybody, even the most gifted, make mistakes.

Also, try to always end a practice session if you’re too tired to concentrate well. Pressing on when you brain is not in an awareness state is like trying to clean a spill using a water saturated cloth. All you’ll end up doing is spreading the spill.

Conclusion

I hope I’ve helped you become a better math student. I appreciate any feedback.

Leave a comment

GameDevelopersWeb.com Merger

I recently decided to merge my other blog, gamedevelopersweb.com, with my personal one. I was tired of having to manage both separately, and I feel like my other interests (biology, web development, learning) could have an overlap with game development.

For those interested, in order not to lose my SEO ranking for gamedevelopersweb.com, I followed these steps:

  1. I exported all the posts/comments/categories from GameDevelopersWeb.com and imported into a local installation of WordPress
  2. I reworked the categories and adjusted the theme to fit my personal website blog
  3. I exported them from my local installation into MichelCarroll.com
  4. Quickly after, I added a .htaccess 301 rule for each page and article on GameDevelopersWeb.com, including the root domain
  5. I logged into Google Webmaster Tools, and advised them that the domain name change had been made

I’ve yet to see the results of this, so I hope the domain change will go seamlessly.

Leave a comment

HTML5 Tower Defense Game using EaselJS – Part 2 – Path to Castle

In the last post, we initiated the project, drafted how the game’s overall look & feel, and drew a red circle as proof that EaselJS is working. In this part, we’re going to render the path to the castle.

The Path to the Castle – Concept

First, let’s give ourselves an idea of what the path is going to need to be, so that we’re better prepared to code it.

Conceptually, here’s what its role is in the game:

  • A logical set of way-points for the orcs to follow in order to reach the castle
  • A logical separation between the orcs and the archers
  • A visual path, so that the player knows where it begins and where it ends

Let’s try to keep the code as simple as possible, so that we don’t get tangled in our own logic. In my opinion, the simplest way to represent a path here is as a set of way-points. Here’s an illustration of what I mean:

The big dots represents coordinate points, which we’re going to represent logically as an array of points. The dotted lines mean that the points are connected sequentially in our game logic to form a linear path.

In order to have a nice background, and to not have to worry about rendering a castle later on, let’s use a static image background in which to display the game on. We need to keep in mind the size of the image when creating/using a background image, because shrinking/growing it might distort the image. I’m going to make my image 640 x 480, which will also be the size of my game stage.

Using my master programmer artistic abilities, I present to you with my map:


(if you find it too embarrassing, feel free to make/use your own)

Notice how I tried my best to keep the width of the path constant, so that we have an easier time mapping the way-points with the image later on.

It’s now time to get down to the code.

The Path to the Castle – Coding

First of all, lets move the image file into our project, and render the map to our browser.

Let’s create a new folder inside the root of our project called images, in which we’ll move map.jpg to:

Then, lets modify our existing code a bit to render the map using an Image class and a Bitmap EaselJS object. An Image object is equivalent to an img HTML tag; Javascript uses it to link to images. The Bitmap object is part of the EaselJS namespace; it’s responsible for knowing what to display and where to display it on the stage:

loadGame = function() {
	var canvas = document.getElementById('main');
	stage = new Stage(canvas);

	var mapImg = new Image();
	mapImg.src = "images/map.jpg";

	var mapBitmap = new Bitmap(mapImg);
	mapBitmap.regX = 0;
	mapBitmap.regY = 0;

	stage.addChild(mapBitmap);
	stage.update();
};


After refreshing, you should now see the map render on your browser.

Now, using an image editor, I’ve determined where our path coordinate points are going to be . Essentially, using the pixel coordinates (x,y), I took note of where each node is situated. Here they are, in Javascript list form:

var coordinates = [
	[200, 0],
	[177, 127],
	[289, 152],
	[480, 100],
	[566, 131],
	[575, 250],
	[362, 262],
	[78, 310],
	[80, 394],
	[288, 400]
];


Using this list, and a simple looping function, lets render lines where the path is going to be, so that we can validate that the waypoints are accurate. Here, I’m creating a line for each pair of coordinates. Each line is a separate shape that needs to be added separately to the stage in order to render correctly.

Think of the Graphics object as the computer’s pen. Here’s what I’m doing step by step:

  1. I change the thickness (setStrokeStyle) of the pen to 1
  2. I move the pen to a new location (point 1), without actually drawing (moveTo)
  3. I put my pen down, and give it a red color (beginStroke)
  4. I draw all the way to point 2 (lineTo)
  5. I commit the Shape to the stage (addChild) before drawing a new one

var g = new Graphics();

for(var i = 1; i < coordinates.length; i++)
{
	var s = new Shape(g);
	s.x = 0;
	s.y = 0;

	var point1 = coordinates[i-1];
	var point2 = coordinates[i];

	s.graphics
		.setStrokeStyle(1)
		.moveTo(point1[0], point1[1])
		.beginStroke(Graphics.getRGB(255,0,0))
		.lineTo(point2[0], point2[1]);

	stage.addChild(s);
}

stage.update();


I put this logic right after the code that renders the map, so that the lines appear on top of the map, and not under.

Here’s what should be the results:

Success! Our coordinates are very accurate!

Conclusion

Now we’re ready drawing orcs, and make them follow the path. This is going to be the next part. Until then, feel free to try it out on your own using what you know so far.

Here’s the source code of my project at this point: Source Code

4 Comments

Latin Roots – The “Pro”s of Learning Them

So I found Memrise a few weeks ago, a website that facilitates memorization. They have a plethora of existing “flash card” sets, ranging from memorizing the numbers of Pi to memorizing the names of trees. Their web application actually makes it very fun and interactive tool.

Among the existing sets is Latin Prefixes & Roots. Since I’ll be studying biology next term at school, I figure learning  Latin prefixes will give me an edge. Rather than memorizing all those terms, it might help me better understand where they come from instead. Good deal.

 

Leave a comment

Command-Based Architecture using Comet

Here’s a picture I found in my archives from last year. It’s a rough sketch of how a game developer might use comet to make a command-based architecture for a multi-player browser game.

I mocked this up while I was getting my hands dirty in NodeJS. If I had to do something similar now, I would probably go with Socket.IO.

Command-Based Architecture using Comet

Command-Based Architecture using Comet

 

Leave a comment