Apr 25, 2015
In the past, I’ve been reviewing myself on a quarterly basis to track my progress against my goals. Here, I aim to inventory my goals and projects, explain how I’ve accomplished them (or fallen short) throughout 2015 Q1, and then elaborate plans, goals, and projects for 2015 Q2. This is largely in response to my previous review.
Note that despite this review being posted so late within April, it only covers the time period of 1 Jan 2015 through 28 Feb 2015. I have not added any additional information about things that have happened after 28 Feb.
I work as a data scientist at a start-up in Chicago.
Get back up to spending 30hrs/wk on my day job. [Outcome: No contest.] (Disclaimer: This is 30hrs/wk with full time-tracking, subtracting out time spent on break, at lunch, in the bathroom, etc. -- it is probably still more than a typical 40hr/wk work week.) In the past, I’ve made time tracking a large component of my personal reviews. However, lately, I’ve changed my philosophy away from that, and for a variety of reasons I no longer think “amount of hours worked” is a sufficiently useful proxy for “amount of work accomplished”. While the question of whether I work hard enough at my day job still looms large for me, I no longer think increasing my amount of hours spent working on it means much. I have begun writing on this in more detail and intend to elaborate eventually on my changing thoughts behind the nature of work.
Learn some data modeling. [Outcome: Successful.] While my work is commonly referred to as data science, it is more accurately considered “data engineering” (and “Data Engineer” is my actual title). This means I do a lot more programming and a lot less statistical modeling relative to a typical data scientist (but still more statistics than a typical programmer / software engineer). I wanted to actually do a data modeling project so I could learn more about the “other side” of data science. I was able to do this, completing a data modeling project throughout January.
Learn Hadoop. [Outcome: No contest.] Our work ended up deprioritizing Hadoop usage, so this skill was no longer relevant. I’ve become more careful with crafting learning goals, which I elaborate on in the section “Learning”, below.
What Went Right
I just finished my ninth month as a professional programmer and my fifth month as a Data Scientist. And just sixteen months ago I started studying programming seriously. Each time I do this review I talk about how I’ve learned a lot, and this review is no different.
I feel like it’s hard to know how you’re a better programmer, but a few things stand out as signs of development:
I can look back on past code, see how it is inadequate, and improve it. A public example of this will be seen below when I discuss the EA Survey.
I’ve learned the virtue of making small, modular functions and engaging in functional programming as much as possible. This makes my code cleaner and more readable.
I’ve learned the virtue of using external libraries and not re-inventing existing solutions. But I unfortunately learned this by experience. This is discussed in “What Went Wrong” below.
I’ve introduced two new R best practices to our team that no one else on the team had been thinking about before. I think I’m finally contributing as an equal.
People ask me a lot of questions now, seeking my help.
What Went Wrong
The biggest problem is that I “wasted” a lot of time through tackling work suboptimally.
For example, one project I had was a “big data” project, where I had to handle data with 10M - 100M records. I conceived of a solution in my head and went for it, spending two months building the architecture. However, later I learned of an external library which supported a different approach that was superior and easier to implement.
I had considered this library earlier but dismissed it, having not spent enough time to get to know it. Once I implemented this library, I was able to re-create the needed architecture in only four days. What took two months of my time could have been done in two weeks if I spent a bit more time researching beforehand. (Another reason not to count “hours worked”.)
The other biggest problem I had was mismanaging a data retrieval project by misunderstanding how high priority the project was and by not asking for help when I needed it. I learned that it’s important to ask clarifying questions and understand the priority and process behind projects before doing them. Luckily, for what I’ve been led to believe are unrelated reasons, I’m no longer in charge of any data retrieval projects.
I think these mistakes happen, and if I wasn’t making these mistakes, recognizing them, and learning from them, I don’t think I would be able to say in good confidence that I was improving as a programmer and as a corporate worker. So I’m not disappointed. But I hope not to repeat these mistakes on my future projects.
I made a mistake last quarter in not making sufficiently good goals that would survive three months of change, so this time I’d like to make my goal a little bit more general: Reflect on how I can be a better programmer. My ideal deliverable for this goal would be three things: (a) an blog post overviewing of what I’ve learned programming to date, (b) a blog post with thoughts on how programmers can become better, and (c) a reflection in my Q2 review on how I’ve improved as a programmer.
In September 2014, I spent five hours launching 64or32.com, a dinky advertising-supported application to tell people whether they have 32-bit or 64-bit browsers. It’s earned about $50 since it’s initial launch, but has slowed down to earning ~$0.10 a month, which I take it means the dream is over.
Since Sep 2014, I have been working with two friends, fiddling around with several different project ideas, even getting two of them close to a possible launch. But ultimately we put both of those two on hold to pursue another idea, and then put that idea on hold to pursue yet another idea.
Unfortunately, I don’t think there’s a lot we can safely say. I can say that one of the two projects is open sourced (i.e., free) modeling software. The other one of the two is not one that I can talk about publicly out on the open internet yet. I do err on the side of transparency, and I do hope to be able to talk about both projects more freely by the end of Q2.
Launch something. [Outcome: Did not happen.] By the end of 2014, we had two projects that both were only a few weeks away from a potential launch. However, we think that those two projects are not as high-value as the projects we’re working on now. However, one of the two projects (the open source one) I hope to have launched by the end of Q2. The other project will take longer to launch.
Move monthly revenue from $6/mo to $12/mo. [Outcome: Success] Surprisingly, this did actually happen, though not in the way I expected. We’ve successfully moved monthly revenue from a one-digit figure to a two-digit figure!
What Went Wrong
Entrepreneurship is really messy and there is a very large gap between practice and theory, especially when the theory is really just anecdotes about other people’s practice.
The hardest thing has been a lack of planning, with too many ideas that seem good and not enough execution. This led us to move between idea to idea, getting a few ideas close, but not doing the final push to launch any of them.
While I guess it’s good no one can accuse us of a “sunk cost fallacy”, this seems generally like a bad habit to be in, and I’m nervous that we’re making a mistake by not following through. Frequently in developing our potential products we enjoy the coding and solving hard problems, but then turn away from the “boring” development side of things, like writing documentation, talking to potential users, etc.
This ties in somewhat with our other problem -- while I’ve personally read countless articles and books about taking a “minimum viable product” approach, I tend to forget to actually do this in any of the actual work. We keep adding cool features because they’re fun and we’re sure they’ll be useful, without just getting the project out there first and asking potential users.
Also, there have been times when I’ve been a bit too excited about these ideas without potentially thinking them through. This has led me to scare a few people with notions of quitting my job when, upon further reflection, I don’t really have any basis (or desire) to do that yet.
What Went Right
I put “What Went Wrong” before “What Went Right” because I’m really tempted for this section to say “Nothing”. But that’s clearly not the case. Overall, I’ve been impressed with our team and our ability to make significant progress quarter-to-quarter while still all having full-time jobs and other priorities.
Launch the modeling software project as an invite beta ASAP. We really have to get this in front of potential users and reflect on their feedback.
Launch the modeling software project as an open beta before the end of Q2. This is a bit of a stretch goal, but we think we can make it.
See whether the other project is actually launchable. This mostly involves legal due diligence. While we might not get a clear “yes” or “no” answer, I’d like to be a lot more confident with it than I am now.
Move total revenue to >$1K. Our current total revenue stands at ~$250 and I have some good reasons to expect some growth. I think total revenue is potentially a better figure for my purposes than monthly revenue, since things are still fluctuating a lot. But let’s see what we can bring in over Q2.
What Went Right
The point of this is not to review the impact of Charity Science, but my personal impact with respect to Charity Science. The biggest thing I did in Q1 for them was host both Joey and Xio here in Chicago. We had a good time as a mini-vacation, but we also arranged two speaking opportunities -- one where all three of us were on a panel at Chi-Fi and Joey spoke at the Chicago Sunday Assembly. In both cases the audience was small (~9 people and ~22 people respectively), but in both cases the talks were well-received.
(Also, I must admit that really it was Marcus Davis who did all the arranging for the two talks and all I had to do was figure out how to get Joey and Xio there.)
What Went Wrong
I considered “ramping up my involvement” in Charity Science for Q1, but attempting to do so failed and ended up creating some burnout. As anyone can see from scrolling through this list, I’m attempting to do too much, and I really need to cut back. I personally feel less irreplacable at Charity Science than on my other projects, so it’s probably for the best that I just remain where I am as a board member, advisor, and donor.
Avoid taking on too much responsibility in Charity Science. So far, I’ve begun by being candid about what I can and cannot do and avoiding taking on additional work. I haven’t committed to a particular hour target yet (especially now that I dislike hours targets), but that might be a good idea if I still find myself doing too much. I expect to evaluate this goal by asking “is Charity Science work stressing me out?”, which the answer is currently yes, but hopefully will be no by Q2.
What Went Right
First and foremost, I finally finished it.
Second, while doing the EA survey did not teach me programming, per se, it definitely served as a good benchmark for my development as an R programmer. When I first started the survey analysis back in September, I wrote quite clunky script. Eventually, by November I learned a little bit more about what I was doing and created surveytools. But later in January I learned about thedplyr library and realized that surveytools was duplicative and inferior, and I wrote surveytools2 based off of dplyr.
What Went Wrong
I don’t think I’ve ever fallen prey to the planning fallacy quite this badly. This project had significant time overruns and I feel bad for not being able to complete it in a timely manner. But I definitely learned a good lesson about how long analysis takes, how to do it more quickly, and what I can and cannot take on given my workload.
Luckily, the next time the analysis happens, it should go much more quickly, because the hard problems (in R analysis, though not survey methodology) have been largely solved.
In Q3, Jason Ketola and I designed a study to determine the effectiveness of advertising pro-vegetarian videos on Facebook. In Q4, Jason and I teamed up with Mercy for Animals to implement the study, acquiring $75K in funding.
Since the study is ongoing in the field collecting responses, I did not have to do anything for this in Q1. I also don’t expect to have to do anything in Q2.
I’ve been trying to learn R programming (to get better at my job), mathematics (to get better at my job), and Chinese (just because it’s fun).
What Went Right
I actually sat down and learned stuff!
What Went Wrong
While I made some progress on these skills, the progress was neither as fast or as steady as I would like. I did well in January, but things fell off a cliff in February and March as I got less motivated and more busy. I think I just need to work on setting more realistic goals.
Finish Advanced R
Finish R Packages
Master >75% of Khan Academy’s Algebra II
Have >300 words in long-term memory on Memrise.
Learn >75 words on Skritter.
Finish the Chinese 101 Coursera class.
Complete >5 more lessons in Rocket Chinese.
Over 2015 Q1, I read 11 books and tried reading two additional books that I ended up not finishing. I wrote up a new reading list with more details. Only one of these 11 completed books was in print, which just goes to show how powerful Audible is for clearing through my reading list.
I also cleared out all the articles in my Pocket, which I consider a significant accomplishment.
Over Q2, I’d like to transition to a phase where I consolidate what I’ve read instead of seeking to read more (though I do still plan on reading more). The main deliverables for this will be (a) expanding the summaries of books that I’ve read and liked (including writing my own where necessary), (b) completing an organized document with all of my link posts, and (c) completing more links posts.
Previously, I had not reflected on my social life as a part of these reviews, but I think that’s a mistake. I don’t want to go into too much detail because I do want to protect the privacy of others and have some privacy myself. I’m also struggling in this area, deciding what I do and do not want from my social life, how I do and do not want to review it, and what I do and do not want to share publicly.
What Went Right
My girlfriend: I think our relationship overall grew much stronger over these past three months. I’m grateful I was able to see her in both January and February (we have a long-distance relationship), which really helps.
My parents: I’ve gotten into a much better habit of calling my parents regularly, which I’m excited about!
My brother: I’ve been keeping in touch regularly with my brother as well. I also got to visit him in late March.
Local friends: I’ve been making more of an effort to get to know my co-workers. I also have one friend from college who also lives in Chicago, who I’ve been seeing regularly.
Distant friends: I’m glad that I got to see a good amount of my college friends in late March.
What Went Wrong
My girlfriend: Quite a lot of the time this past three months I ended up planning rather terribly, neglecting my girlfriend and not considering her needs. I’m saddened by having hurt her that way.
My brother: My brother and I have both agreed that we want to stay in touch more consistently, but we haven’t developed a consistent schedule yet. Of course, perfect consistency is unrealistic, but I’d still like to improve by being in contact more.
Distant friends: There’s no regularity to our contact, and I need to fix that. Outside of visiting them once, I have not been doing well at keeping in touch.
Connecting with people I know less deeply: I’d like to be in somewhat regular contact with people I know less deeply from college and my past; people who are more my acquaintances instead of my friends.
My big goal here is to figure out a system for improving my consistency in communication with everyone I want to be in touch with, as well as sorting out who I want to be in touch with and at what quantities.
More important than nearly anything to a healthy and productive life is really nailing the core three: exercise, sleep, and food. Unfortunately, I’ve struggled with these three so far my entire life.
What Went Right
I’m happy with the health of my current, vegetarian diet. I have taken the words of wisdom of Michael Pollan to make it not much more complicated than to “Eat food. Not too much. Mostly plants." I get a good amount of fruits and vegetables in my diet every day and don’t eat too much junk food. There’s probably room for improvement, as I could stand to eat less and to maybe eat less bread. But I’m happy with where I am.
What Went Wrong
While food is on target, exercise and sleep are not.
I’ve been struggling with going to sleep at a reasonable time (i.e., before midnight), getting up at a reasonable time (i.e., before 9am), and sleeping a reasonable amount of time (my problem is actually oversleeping, not undersleeping like most Americans). This has created a bad morning routine which has worsened my work life, and, from there, worsened my personal life.
Exercise has not been great either. I’ve kept things steady at 1.4hrs/wk, which is the same as 2014 Q4, so at least things have not gotten worse. And by many measures, I should be exercising enough, as I walk 30 minutes every day commuting to work (12-15 minutes both ways), and frequently walk (or even run) up the six flights of stairs to my apartment. However, I’d still like to have a routine of going to the gym at least once every week, not to mention twice or three times. I’d also like to do the same with running. I think both are reasonable goals for getting more healthy.
For my goals in this area, I’d like to create the reasonable goals of (1) getting to work by 9:30am more often than not (since this would mean I’m waking up well enough and having a decent morning routine) and (2) going to gym at least once a week and running at least once a week.
If I’ve been struggling with both sleep and exercise since the beginning, it’s a bit silly to just re-commit to the same goals every time. Instead, I must follow the rule which asks “If you’ve failed in the past, what will be different about this time?”. To this, I answer:
I’m committing to much less difficult goals than usual. For example, going to the gym once a week instead of 2-3 times; getting to work more often than not by 9:30am, rather than all the time by 8am. This should build me a success spiral toward where I really want to go.
I’ve put things under the watchful eye of Beeminder. While many times I end up paying money on Beeminder, at least it lets me notice and feel the sting. For exercise, I’m beeminding time spent both in the gym and running, and for sleep I’m beeminding both bedtime and the time I get to work. (I’m also beeminding my diet, even though I think that’s going ok.)
I’m going to consider some secret weapons. For sleep, buying a wake-up light. For the gym, trying to go regularly with a friend. For diet, preparing food in advance.