Skip to content

Development Workflow Upgrade for kpc.com

A day off means an opportunity to make progress on a kevinpaulconnor.com redesign. The site (THIS site, save the extremely unlikely event that someone has stolen my content) is old and busted, having been designed and composed by an inexperienced yours-truly, five years ago. The first order of business, before the real work can proceed, has been to update the amateurish development workflow that exists on the site. See previous (quite previous) updates here and here. I’m happy to report that I was just about able to wrap things up today, and will soon be able to move foward with real improvements.

kpc.com code has been pushed into a (previously established, private) git repository on github.

Previous posts describe XAMPP as a way to get a local development environment going, but it was always too hard to configure and keep up to date. Enter Vagrant, which is a wrapper for Virtual Box that makes stable local dev environments effortless. Vagrant is written in Ruby, and depends on dev-ops environment replication tools Puppet or Chef for module configuration of a core Vagrantfile that is like a makefile. I forked a guy’s LAMP Vagrantfile (repo) for my instance here. You will want to muck around with the Vagrantfile to make sure that it uses the appropriate local directory as its web root-I didn’t push my changes back to the public repo. It’s as easy as:

vagrant up

…in the directory with your Vagrantfile to spin up your instance, or just as easy to suspend or shut down gracefully.

With Vagrant, I can point the web root of my local LAMP instance at my local kpc.com git repo, and edit files under version control, on the fly, and see the results in a web browser pointed at localhost:$PORT (8080, if you’re using my Vagrantfile). Nice! No more editing a file, waiting for it to upload to the server, and then wondering what you might have changed. The Vagrant Getting Started Guide was useful here, as was this blog post.

I am experimenting with the Netbeans IDE today, at the suggestion of a coworker. I had been using the Tower git client for all of my git work. I’m pleased with it, but it makes different assumptions about the status of merge commits than does the Github GUI client, which others on my team use, and that was causing them some problems. I wasn’t able to locate a copy of the Github client for OSX 10.6, which I am stuck on at work for the next little while, so I’m switching over to the command line for my git work. Netbeans automatically and seamlessly updates changes in the file that you’re working on relative to the local git repository, a killer feature that I’ve really been enjoying today in lieu of the GUI git interface. I’ll be switching over to it for most of my work, I think.

For the databases, NearlyFreeSpeech allows remote MySQL connections only through an SSH tunnel, so that needed to be set up before DB access from Vagrant would be possible-this was a little conceptually tricky because you first need to ssh into the Vagrant instance and set up the tunnel from there. This blog post was helpful-I know I’ve had trouble setting up the tunnel for NFS before, hopefully it will stick this time. You wouldn’t want to (or be able, per NFS policy) run production SQL queries through a PHP CLI SSH tunnel, but for this instance you can set up the tunnel in a terminal and then just go to town. I’ll have to maintain dev/prod DB authentication differences in my code base-that will be one of the few concessions to the differing environments.

For deployment, I had previously set up a staging account on NFS with its own separate MySQL instance. I added staging and production repositories as remote branches to my local git environment, then followed this method for easy deployment: added a post-receive git hook with a hard reset, which will return the server-side repository to the head with the latest commits that I’ve just pushed as they come in. Now I can do:

git push staging master

…on my local git instance to verify server-side in case the minor differences between Vagrant and NFS are catastrophic, then

git push prod master

…if all is well.

I can now work locally and instantaneously on versioned files in a nearly identical local environment to my production environment (and it could be easily and maintainably identical if I dug down into it, but I don’t think that will be crucial for my uses). I can push my changes to a remote development repository on github, as well as push my local environment configuration changes to a remote repository-if my local machine goes down, nothing will be lost. I can get one-step deployment to staging or production.

I’m still missing a plan for database replication, but that’s not a huge priority for how I use these databases. With this updated deployment workflow in place, I’m ready to get this website in shape! Probably not the part that you’re actually reading, though-I’m still a fan of this WP theme.

How to configure vim

Everyone has their own opinion about this, but I found this post to be useful.

Where to declare javascript variables

I’ve been reading the new edition of Flanagan’s “Javascript: The Definitive Guide”, straight through, and I thought I’d make some posts about what I’d learned.

In javascript, variables are implicitly declared at the beginning of the function where they are used. So, in the following block,

global = 'global'; 
 
function f() {
   console.log(global);
   var global = "local";
   console.log(global);
}
 
f();

The first console.log will log ‘undefined’ instead of ‘global’ even though the local variable named global has not been defined, because the javascript intepreter knows that a local variable named global will be defined within the scope of function f and implicitly declares it at the top of that scope. Good programming practice as I had understood it is that you should declare variables as close to where they are used as possible, to narrow their scope as much as possible. With this realization, for javascript, variables should be declared at the top of their scope, to make the readable code match the scoping that the language is doing behind the scenes.

.gitconfig and core.autocrlf

A few notes on the git things I dealt with today:

I updated my .gitconfig file to exclude everything in my wordpress directory from source control except plugins and themes. Since these were already in my repository, I need to remove them-git rm -rf –cached $files will do the trick. Note that if you leave off the –cached parameter, the files will be removed from the filesystem in addition to the repository. Care is required.

I had an upgrade to the headspace2 plugin on my wordpress installation today. When I tried to commit these changes to the repository, I got the following error:

“fatal: LF would be replaced by CRLF in blog/wp-content/plugins/headspace2/2.3/jquery.js”

I changed my core.autocrlf from input to false and re-added; this stopped end-of-line coercing and allowed me to get the file into version control. I’ve read that core.autocrlf should be on input for almost all use cases, so I’ve changed it back. I’m not sure what the long-term repository ramifications of this bit of hackery are; the headspace plugin will be a decent test case, and I’ll keep an eye out for problems.

Time for Some Direction

Over the last month, I’ve been thinking a lot about how to improve kevinpaulconnor.com, but I haven’t thought very much about what that means. Today that changed; I sat down and created a mission statement and some deliverables for myself with regard to improving kevinpaulconnor.com. I’ve recently made some decisions about basic tools to fill some holes in my expertise: version control system, text editor. I’ve decided how I want my development environment workflow to fit together. Now it’s time to decide how to apply those tools.

Mission: Kevinpaulconnor.com should highlight my vision and project my professionalism as a midlevel web developer to site visitors, in terms of both Design and Content and Markup and Code.

    Design and Content: Professional presentation with the existing “sheets of notebook” paper framework as a guiding principle: a “freshening”, not a redesign

  • Inspirational guiding principles: add glitz and polish, 1950’s diner, neon, Vegas. Bolder, but still minimalist
  • Improve font choices: move away from ‘typewriter’ design
  • Instead of ‘sheets of paper’, edge towards ‘blank canvas’ vision of each page. Less structure with more vibrant presentation. Maitain rigidity, but with less boxiness: few curves, but see angles rather than lines
  • Existing colors are flaccid. Move from gray to white, and improve dim reds and blues and blacks
  • Professionalize content and message. More serious, more confident. Less purposefully obfuscatory
  • Integrate all pages into the new design: be rigorous about cohesion of presentation
    Markup and Code: Professionalize across the board

  • Restructure markup and css to reduce complexity and increase re-use
  • All markup standards compliant with html5 and css3
  • Improve browser compatibility: achieve consistency
  • Integrate sophisticated javascript to demonstrate ability, as appropriate. Imagine features for this purpose.
  • Choose a new server side language/framework. Leaning Ruby on Rails at the moment. More on this next time
  • Mobile: Make kevinpaulconnor.com shine on iOS and Android devices

Progress on the Dev Environment

Going over (or through!) all of the usual hurdles when you’re setting up something new for the first time. All sorts of little things to figure out, which you don’t even know need figuring out until you get there, adds up to everything taking longer than you think. I sat down this morning thinking that I’d get to start looking at all my old, crappy kpc.com markup and code finally. But instead,

-By default, xampp wants the document root of your website to be in the xampp/htdocs directory. That’s lame; I don’t want to modify my whole local filesystem structure just to please xampp. I changed the document root to the place where I was already storing kpc.com site files in the xampp httpd.conf (make sure that you change both the document root and the default directory). The ideal way to do this would be to instead set up the directory as a virtual host, so that I more easily manage multiple websites, but I’m not planning on managing any other websites besides kpc.com on this xampp installation in the near future, and it’s one less configuration change to muddle through in the present.

-When you change the document root, xampp will throw 403 errors unless you give world execute permissions, not only the specified web root and site files, but to every folder above the web root in your filesystem. It took a bit of time to figure out why this was happening.

-By default, xampp will serve .php files as plain text. To change, add “AddType application/x-httpd-php .php” to the section of the httpd.conf. You could also add this handler to the file specified in the TypesConfig directive, etc/mime.types by default. (As always when making any httpd.conf changes, remember to restart apache for the changes to take effect)

-It’s been on my to-do list to make a definitive decision on a terminal text editor for years. I’ve been using nano, because…I always have? That’s not a very good reason. I decided to go with vim today: I want my fingers on the home rows as much as possible, I like the concept of vi’s distinct editing modes against emac’s chords, vim adds some nice features including syntax highlighting, it’s going to be installed on anything I ever use, it’s super lightweight, and I don’t care about emacs’ GUI advantage-if I want a GUI, I’ll just kick up Text Wrangler. Pico is appealing, but marginally less universally available and significantly less beloved. A great way to get started is typing ‘vimtutorial’ into your shell and work through those exercises. It won’t give you the power user commands that make things really speedy, but it will get you enough along the way to enable you to use it. after that, :help vimrc-intro and :help user-manual, from within vim, are recommended-these are on the to-do list for next time. I’m also looking for a good vim keychart that I can print until the keystrokes get ingrained-recommendations welcome!

-I started working on a .gitignore file. I won’t want to, for example, track WordPress package changes in my git repository. I haven’t made too much progress in thinking about this…today was mostly a server configuration and editor day. An investment in future productivity!

Git Thoughts, Git Resources

We use Perforce at work, and that’s my only serious version control experience. Some thoughts about getting started with Git in the context of my at-work workflow:

-Both Git and Perforce have a staging step, but Perforce’s step happens prior to file edits, while Git’s is after file edits. This results in a time-wasting checkout of any file that I ‘might’ want to edit: since I can’t make any edits until I go through a checkout step, I usually might as well do it if I think I might have to edit a file. So if I’m trying to resolve an issue and I’m not sure where it is, I wind up having a bunch of unedited files checked out when I’m finished, that I then have to go through and check back in, purposelessly-or I wind up opening a bunch of files but not checking them out, and then having to go back and check them out in a separate step when I know that I need to edit them. Git’s philosophy is, keep source control out of the way until you need to deal with it. Perforce is in my face from the very beginning.
-Git is centrally concerned with changes to files; Perforce is centrally concerned with files. This is surely a change that will have wide-ranging impact on the way that I conceive of my projects. I’m just getting started, though, and since I barely have any real work in a Git repository yet, it’s hard to say exactly what those changes will come to. It’s something that I’m going to keep in mind as I continue working.
-git commit –amend allows you to add changes to an already submitted changelist, rather than submit a new one. An example of how this would be useful at work: our css/js is usually served as static assets from our cdn; so you have to remember to make changes to the separate files that bust caching so that your css/js changes are picked up. It’s easy to forget to do this; then you get two changelists, one with your actual changes, and one with the static asset cache-busting, that you have to relate to the original list via a comment. This is somewhere between pretty lame and very lame depending on my mood; with git, it would be a non-issue.
-Branching for us with Perforce involves much handwringing: managing branches is difficult for us and it’s not something we do lightly. With Git, branching is trivial. Precisely what impact that has on workflows (besides “a big impact”), isn’t clear to me quite yet, but it’s something that I’ll be keeping an eye on.

Git Resources

I worked through Edgecase’s Git Immersion lab, and it’s been the most useful resource for me so far. I think that I learn best while my fingers are doing something as I hear about what it is that they’re doing. I think a lot of people learn best that way, and that’s what you get to do with that tutorial.
I found that link through Git Ready, which has the best collection of Git resources that I’ve found so far. The other one that I’ve worked most of the way through from that site is Pro Git; it’s pretty good, but aimed a little shallowly for what I needed. I’ll be working through a lot of the other links on Git Ready and I’ll have more to say about them as I go. The other noteworthy resource, for specifically getting started on managing kevinpaulconnor.com development with Git, is Abhijit Menon-Sen’s guide. I think that one is a good jumping off point for how to set up your Git hooks. This is not a use case where Git is supposed to excel (a website managed exclusively by one person), so I’ll try to pay attention to places where things seem a little sticker than would be ideal.

Workflow changes at kevinpaulconnor.com

Kevinpaulconnor.com is approaching three years old. I’ve learned a lot about development since I created the site; it’s time for a design and development freshening, with the twin goals of making the site more visually appealing while also making the code more clear and maintainable. Before that makes sense, however, my development process for the site must be improved and modernized. Currently, the development process for the files is to FTP into my hosted server and edit live files. This process has a few serious drawbacks:

-FTP is an insecure protocol-my username and password are sent in the clear across the network, meaning that anyone can gain access to kevinpaulconnor.com if they are sniffing with the right tools in the right places.
-If (when) I make a mistake with my markup or code, that mistake is immediately propagated to the live site, impacting user experience or bringing the site down altogether.
-Changes to my files are lost forever-if I decide that a previous revision of a file was better than one I’m currently using, too bad for me.

I’ve inserted the kevinpaulconnor.com files into a git repository, downloaded and installed XAMPP to my local machine, and created a staging.kevinpaulconnor.com subdomain. The new workflow will be:

-Design and develop locally against XAMPP on localhost. Commit changes into my local git repository. This repository is synched with staging.kevinpaulconnor.com, which is protected from non-me pageviews by http authentication. Push changes to the staging domain, and inspect for errors. If all looks well, push changes to the live kevinpaulconnor.com domain.

This workflow solves my three issues above:

-Git pushes will now handle all my networking traffic between local, staging, and live environments, securely.
-By designing and developing locally and then testing on the staging site, I can catch coding errors before they propogate to the main site.
-Git will keep permanent copies of all my site revision changes-if I decide to revert to a previous edition of a file, git makes this easy.

Today and last weekend I’ve been working on implementing these changes: kpc.com is currently in a local repository, staging.kevinpaulconnor.com is up, protected, and set up to accept one line git pushes from the local repository, and I’ve installed XAMPP and the developer tools. For the moment, the live kpc.com site hasn’t been touched. Over the next few weeks, I’ll be thinking about how to accomodate this blog in the new workflow (particularly the database), setting up XAMPP to enable purely local design and development, and integrating the live site into the git workflow. Exciting!

Cot’s/Yahoo GM Script revised for 2011

“Updated for 2011 Season! Pulls MLB Service Time from Cot’s blog and displays in player title tag on Yahoo Fantasy Baseball! “

This script is extremely useful for a small subset of fantasy baseball players, and otherwise…not so useful.

I refactored the code completely this year. It’s much faster, and it now caches the information that it gets from Cot’s in your browser-improving subsequent load speeds considerably.

Android: Environment Configuration

  • My Android emulator window doesn’t respect the screen resolution on my Macbook-so the emulator screen extends through the bottom of my window, and can’t be resized. To fix, follow this stack overflow answer. You may have to tweak the scale parameter, but the suggested .75 worked fine for me.
  • The documentation here telling you how to log errors is good (generally, you should read all of that documentation, but I recommend diving in first so that you have a little bit of context. I’m planning on getting these sample projects from the book up and running and then reading the docs before I build my first ‘real’ project.) The docs are unclear about how to actually SEE the logs though-from the main perspective, go to Window->Show View->Error Log.
  • With regard to the above, experiment with the available perspectives and views from the Window method to find all of the information that you might want. There is a lot more going on than what the default views in Eclipse will show you.
  • The modified TodoList project from Chapter 3 of the WROX book will default to Chinese characters. From googling around it appears that this is a common problem-it may even be a problem with shipped hardware, rather than an emulator problem! Follow this to fix.