devuxer

19 December 2014

Delimited column selection

It recently occurred to me that column selection is nice, but it could be awesome.

Here is what I mean by column selection:

In most GUI editors, you perform this type of selection by alt-dragging the mouse down and to the right.

If all you wanted to do was add virtual after each public, this would be very useful.

But what if you wanted to strip away public plus the data type? For this, you would want the selection to go word-by-word instead of character-by-character as you alt-drag the mouse to the right.

To illustrate this concept, I created an animated GIF. (In case your curious how, I used PowerPoint to create the slides, saved them as PNG, imported them as layers into Paint .NET, and saved out the image as an AGIF using the Animated Image plugin.)

I see this feature being useful for any situation where you have a bunch of similar declarations and you want to strip away something from each line or insert something new between the same two words of each line.

While writing this post, I began to wonder if delimited column selection already exists. I was thinking Sublime Text might have it, but I wasn't able to find it during my brief test drive. The default Visual Studio editors don't seem to have anything beyond standard column selection, but I did come across the MultiEditing extension, which Scott Hanselman blogged about in 2013. While it sounds quite useful (I think I'll install it), it wouldn't be quite as easy for the use cases I have in mind.

Anyway, if you'd like to see delimited column selection in Visual Studio, I just posted the idea to Visual Studio User Voice.

Read more...
15 February 2014

Why The Mercurial ZipDoc Extension Fails For Excel Files

The problem

I tend to work on small projects with small file-based databases (usually SQL Server Compact Edition). I find it very convenient to use Excel files to create the seed data for these small databases. I even have a little library that uses a combination of EPPlus, Entity Framework Code First, and good ol' ADO.NET to basically convert any properly formatted Excel file into a SQL Server Compact database.

Given that these Excel files are part of a project and change over time, I like to keep them in source control (in my case, Mercurial). Unfortunately, given that Excel files are not simple text files, it's not easy for Mercurial to determine what has changed from version to version. This makes it basically impossible to visualize what has changed, and over time, it can bloat the repository because the deltas are not stored efficiently. I've always wondered if there was some way to do this better.

The solution?

After a little Googling, I came across the Mercurial ZipDoc extension and downloaded it. After a little more Googling, I realized I didn't need to download it, since it was already included with TortoiseHg. ZipDoc is supposed to allow for more efficient delta compression when committing Open Office format files (such as docx and xlsx).

The test

After making the appropriate modifications to mercurial.ini file, I performed a small test with a sample Excel 2010 file. My file consisted of one worksheet with the following content:
```
      |    A     B     C     D     E     F     G     H     I     J
------------------------------------------------------------------
  1   |    1     1     1     1     1     1     1     1     1     1
  2   |    2     2     2     2     2     2     2     2     2     2
...   |
10000 | 10000 10000 10000 10000 10000 10000 10000 10000 10000 10000 
```
In other words, columns A to J of rows 1 to 10,000 were filled sequentially with numbers.

When I saved this file, its size on disk was 318 KB.

My test was very simple:
1. Create a repo with just the Excel file (Book1.xlsx) in it.
2. Commit.
3. Change the value of cell A1 to 10.
4. Commit changes.
5. Change the value of cell A1 back to 1.
6. Commit changes.
After each commit, I noted the size of the .hg/store/data folder.

The (dismal) results

Here are the results:
```
                   ZipDoc      ZipDoc
Commit Message    Enabled    Disabled
-------------------------------------
Initial commit     247 KB       32 KB
Change A1 to 10    493 KB       59 KB
Revert A1 to 1     738 KB       87 KB
```
I was expecting ZipDoc to result in a larger initial data file size, which it certainly did, but I was not expecting it to increase so rapidly with only very minor changes to the Excel file. If fact, with ZipDoc enabled, the data file size grew at virtually the same percentage rate as with ZipDoc disabled (but starting at a much higher initial size).

This basically means that ZipDoc was not performing well at all, at least for my particular test case.

The explanation

For the Excel file in question, the structure of the zipped archive can be revealed by adding ".zip" to the name of the file and extracting the contents.

The result is this:
```
_rels
    .rels
docProps
    app.xml
    core.xml
xl
    _rels
        workbook.xml.rels
    theme
        theme1.xml
    worksheets
        sheet1.xml
    styles.xml
    workbook.xml
[Content_Types].xml
```
I used Mercurial to analyze what files actually changed between two different versions of the Excel file. The answer was just xl/worksheets/sheet1.xml.

Opening this file, the reason for the dismal performance of ZipDoc became instantly obvious: the entire file was contained on one line (it was essentially compressed or "minified", containing no unnecessary white space). Since Mercurial compares files based on which lines have changed, it will always conclude that the entire file has changed. Hence, ZipDoc will never be able to do its job properly under these conditions.

When I actually formatted (prettified) sheet1.xml using Visual Studio's XML editor before committing it, Mercurial was able to find the one small difference between the two versions of the file, and it was able to store the delta very efficiently.

The conclusion

As long as Excel saves the component XML files as one-liners, ZipDoc won't be of any use, and will actually do more harm than good.

I'm not sure whether ZipDoc is being actively developed (the last update was in 2011), but in the hopes that it is, I posted an issue to the project's Bitbucket repository.

I'll try to keep this post updated as I learn more.

Read more...
13 January 2014

Serving up a website from a console app using Nancy

Have you ever wanted to build a desktop application that runs in a browser? As this post explains, Nancy makes it pretty darn easy--as long as you follow the right approach.

Use case

My use case was simple. I wanted to build a web application, then I wanted to give it to coworkers at my company as well as some external clients to run on their own machine. I didn't want them to have to worry about starting up a web server or dealing with any configuration. I just wanted them to be able to copy over a few files and double-click an EXE. In other words, I wanted it to "just work".

Self hosting Nancy

When I mentioned what I was trying to accomplish on JabbR, GrumpyDev suggested (in an entirely non-grumpy manner, I might add) that I try self hosting Nancy. All I really understood about self-hosting at that point was that it was an alternative to ASP.NET hosting. Reading the wiki page (and peppering GrumpyDev with several questions) provided a bit more insight, so I decided to give it a try.

This is almost awesome!

The first thing I did was create a console app. Then I added the Nancy.Hosting.Self NuGet package. Then I added a module. So far, so good. Then I went to add a view, style sheet, and JavaScript file. That's when I got my first inkling that I was going a little against the grain. This being a console app, not a web project, the Add menu looked like this:

Instead of like this:

Of course, all these files could still be added, I just had to go into the Add Dialog to conjure them up. So, that's what I did. Not a big deal.

After fiddling a bit with the configuration and realizing that, oh, all build settings for all content files (namely, views, images, style sheets, and scripts) need to be "Content-Copy Always", I was up and running.

It took a minute to sink in, but this was actually a big deal. I was viewing a web page in my browser, but there was no apparent web server running. No IIS, no IIS Express, no Cassini. Pretty sweet.

Then, I attempted to edit a view. After saving the file, I reloaded the browser. Nothing changed. I checked to make sure Visual Studio was in Debug mode, not Release (since Release mode causes Nancy to cache pages), but that wasn't the problem. Then it dawned on me, since content can't be picked up unless it is copied to the appropriate bin folder, I needed to re-run the app.

That worked, but it was, again, not quite the usual workflow. Still, I reasoned, "I can live with this."

Then, I started to notice some issues with ReSharper: it was being a little more picky than usual about how I wrote my relative paths, and a number of items seemed to have disappeared from the intellisense menu when using the JavaScript editor. It even threw some inspection warnings for using window methods (such as alert()), which prompted me to write a StackOverflow question in frustration.

None of these problems were deal-breakers, but I started to think, there must be a better way.

A better solution?

After pondering a couple different ideas, I decided to try a dual-project approach. I knew that, when it came to actually writing HTML, CSS, and JavaScript, I wanted a good ol' web project, but when it came time to deliver the results to others, I wanted a nice, portable, self-hosted, console app. So, I created one "ASP.NET Empty Web Application" and one "Console Application", and added the appropriate Nancy NuGet packages to each. Next, I put all modules, views, and content files from the original project into the web app.

Then, I got to what I thought would be the tricky part. How do I get the files from the web project to the appropriate folders in the console app? It turned out, I needn't have worried. I just added a reference to the web project from the console app and made sure, as before, that all views and content files were set to "Content-Copy Always". As for the module, it turned out I didn't need to do a thing. Nancy is smart enough to scan both the primary project as well as referenced projects to find modules.

Details

A couple notes:
1. After installing the Nancy.Hosting.Self NuGet package, I modified my Program class to look like this:
```
internal class Program
{
    private static void Main()
    {
        var configuration = new HostConfiguration
        {
            UrlReservations = { CreateAutomatically = true }
        };
        using (var host = new NancyHost(configuration, new Uri("http://localhost:1234")))
        {
            host.Start();
            Process.Start("http://localhost:1234");
            Console.WriteLine("Press any key to exit.");
            Console.ReadKey();
        }
    }
}
```
  - The UrlReservations bit was to prevent an AutomaticUrlReservationCreationFailureException, as described in the wiki.
  - Process.Start() simply opens the home page of the web app in the default browser.
2. Just in case you were tempted to use the same port number for both apps, don't do this (I tried). It will fail, even if you never run both apps at the same time.
tl;dr;

If you want to build a web application for individual use, and you want it to be portable and as easy to run as any desktop app, use a Nancy self hosted console app for deployment, but do your web development and debugging with an ASP.NET-hosted Nancy web app. It's barely any extra work to set up this way, and it will ensure all your web development tools work as designed.

Read more...
15 December 2013

Powered by Sandra Snow

Welcome to my new blog. I've wanted to have a blog for a while, but I hesitated because I thought it would take a lot of effort to get started.

Now that I've done it, I can say, yep, it really was a pain!

There are three reasons creating this blog was hard:
1. I started from nothing.
2. I wanted a solution that would give me full control over my content.
3. I didn't want to pay for hosting.
Going from zero to blog in...many hours

I had to make a number of tough decisions along the way. Here are some of them:
1. What should my domain be? devuxer.com
2. Which domain registrar should I use? Namecheap
3. Who should be my web host? GitHub
4. How will I generate my blog? Sandra Snow
There were also decisions within decisions. For example, Namecheap offers WhoisGuard free for one year. The idea behind WhoisGuard is that it keeps your personal contact information hidden from anyone trying to find out who owns your domain. Apparently, some feel this type of thing makes it easier for scammers to hide their identity, however, not hiding it can result in spam. I left it deactivated for about three days. Then I got my first piece of spam relating to my domain. That was the end of my experiment with not using WhoisGuard.

Sandra Snow

Once I decided I wanted to host my blog on GitHub, I spent quite a bit of time reading about and fiddling with Github Pages, Jekyll, and Jekyll Bootstrap. I really liked the idea of writing my posts in markdown, then running some code that magically turns them into a blog, but I was getting bogged down in the details of how to actually set everything up. I also kept reading things like:

GitHub cannot afford to run arbitrary code on their servers so they lock developers down via Liquid. Liquid is not programmer-friendly.

At some point, not really expecting anything to come of it, I hopped on JabbR and said:

YU so confusing, Jekyll?

Phillip Haydon responded:

Use Snow

That's when things started to get easier. (Not quite easy, but definitely easier.) Rather than a Ruby solution like Jekyll, Sandra.Snow is a .NET solution. If I needed to debug anything (I did), I already had all the tools and at least some of the know-how. And instead of the dreaded Liquid, I would get to use good ol' Razor templates.

A Snow step-by-step
1. Pretend you are going to use GitHub Pages:
  - Create a new repository in GitHub called your-domain (e.g., devuxer.com).
    - Enter a description (e.g., My blog).
    - Don't worry about a README (I didn't bother with it and nothing bad happened to me).
  - Go into the settings for the repository:
    - Click "Automatic Page Generator".
    - Click "Continue to Layouts" (don't bother editing the page).
    - Choose the MINIMAL layout (actually, it doesn't matter).
    - Click PUBLISH.
  - You should now have a gh-pages branch.
2. "Haha, I was just kidding":
  - Install GitHub for Windows, if necessary.
  - Go back to your browser and click "Clone In Desktop".
  - In Windows Explorer, delete everything except the .git folder from your working directory.
  - Commit changes.
3. Replace all that stuff with the Sandra Snow Template:
  - Browse to the Sandra Snow Template.
  - Click "Clone in Desktop".
  - In Windows Explorer, copy everything (except the .git folder) from the working directory of the template to the working directory of your blog.
  - Run compile.snow.bat to generate all the folders and files.
  - Commit changes.
4. Replace anything that begins with "phill" with your own info:
  - Files you will almost certainly want to edit:
    - Root directory: CNAME
    - Snow directory: about.cshtml, rss.xml, snow.config
    - Snow/_layouts directory: default.cshtml
  - Run compile.snow.bat again to generate the site.
  - Commit changes.
5. Tell your domain registrar to redirect all web traffic to GitHub:
  - Log in to Namecheap (or whoever sold you your domain).
  - Navigate to the management screen for your domain.
  - Click URL Forwarding (or whatever your registrar calls it).
  - Set up your redirect to go from your domain to:
    - Record Type = A (Address)
    - IP Address/URL = 204.232.175.78
    - TTL = 3259
6. Set up Disqus to handle your comments:
  - Create a Disqus account, if necessary.
  - Click "Add Disqus to Your Site".
  - Follow the prompts until you get to a "Choose your platform" page, then stop.
  - Locate post.cshtml in Snow/_layouts and replace *disqus id* with your site's Disqus shortname (e.g., devuxer).
  - Commit changes.
7. Set up Google Analytics and AddThis so you can obtain some some nice statistics on your blog:
  - Browse to Google Analytics.
  - Create a new account pointing to your domain.
  - Open default.cshtml in Snow/_layouts.
  - Replace the **** in _gaq.push(['_setAccount', '****']); with your Tracking ID.
  - Add @Html.RenderGoogleAnalytics("[your-tracking-id]") below the prettify script element.
  - Browse to AddThis.
  - Create a new account.
  - Open post.cshtml in Snow/_layouts.
  - Replace the **** in #pubid=ra-**** with the code provided under "Add to your site".
8. Replace all the sample posts with what will become your first post:
  - Download MarkdownPad, if necessary.
  - Delete all but one of the posts in Snow/_posts.
  - Rename the remaining file with the correct date and title.
  - Open the file in MarkdownPad and commence blogging!
  - Run compile.snow.bat again to generate the site.
  - Commit and sync to GitHub.
  - Look for your updated blog to appear in 10 minutes or less.
9. Do some things I haven't done yet:
  - Edit the theme to give your blog a unique look and feel.
  - Make sure people are actually able to leave a comment.
Conclusion

If you want a blog that's truly your own, it's going to take some work to set up. If your situation is similar to mine, hopefully, I've at least spared you some of the struggle. Regardless, once you get over the hump of setting things up, it should be smooth sailing. At least, that's the idea.

Read more...

The problem

The solution?

The test

The (dismal) results

The explanation

The conclusion

Use case

Self hosting Nancy

This is almost awesome!

A better solution?

Details

tl;dr;

Going from zero to blog in...many hours

Sandra Snow

A Snow step-by-step

Conclusion