Bending the Command Prompt to your will and why I don't use Microsoft's PowerShell

I'm a Windows guy for sure, but I bought a MacBook because they are kind of cool. And I occasionally write code for both OS X (BSD) and Windows and often so use command line tools.

One thing that Windows kind of have been missing is a terminal, with capabilities more similar to that what you get from Unix/Linux (from now on referred to as -nix) based operating systems. Microsoft has noticed this and they believe PowerShell is the one solution to all your scripting and terminal needs. I have a couple of problems with Microsoft's PowerShell and that is why I write this.

Auto-completion

PowerShell auto-completion is completely subpar, not only does it fail to recognize aliases or commands in your path environment variable, it rewrites partial paths to full suggestions which covers your prompt. You end up cycling several suggestions before you run into the right one, or the completion in the worst-case will suggest a complete name, while you only want part of that. It makes me think that the guys who wrote PS did not have any background experience in NIX based operating systems #!/bin/bash.

Output

PoweShell does have a somewhat nice way of pretty printing output from different commands, but it's bloated by too much white space. I'm not very keen on output that will take up my whole screen, I want something which is mean and lean as often as possible. PS fails to achieve this with several commands, such as the basic ls or mkdir commands.

That Microsoft currently does not support SSH is just sad, but if you want to be able to SSH to your Windows computer you can always rely on Cygwin (though, Cygwin is bundled with so much more). Anyhow, this post is about making the good old fashion Command Prompt behaving more like a terminal you're familiar with if you know -nix based operating systems. The Command Prompt itself is actually alright.

Some sugar

Font-rendering capabilities are getting better in Windows, with ClearType you get true subpixel rendering which makes things more readable, so why don't apply this to your terminal. All you need is a TrueType font face with ClearType support. A monospaced font you might know of which is excellent for terminals and code is Consolas. I use it for everything code and it is freely provided by Microsoft.

• Install this font on your computer and fire up the registry editor.
• Goto HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Console\TrueTypeFont
• Add a string value "00"="Consolas"
• Reboot (you'll have to reboot for the font to be available in the command prompt settings dialog)

Commands

What I do is that I, set up a PATH variable to point out a bunch of batch files that reminds me of -nix. I create a file ls.bat and put a single line: "@dir /w" in that file. This way the DOS command ls, will actually give something which is more similar to ls. You can tweak around with this to get everything you might need. (I find "dir /w" a lot more useful than just "dir")

This is just a few tricks I ended up using after realizing Microsoft's PowerShell annoyance. The main reason why I "downgrade" is that the auto-completion in PS truly is worse than the old fashion cmd.exe way. I'm quite certain that there is a way to just replace the way this is done in PS but I will not waste my time pursuing it. Cygwin does come with almost everything you need but if you don't want to install Cygwin, you could make do with these tricks.

I primarily use the Command Prompt for svn, because I loath TortoiseSVN. And I wrote this in hope of some realization from people using PowerShell, in that it falls short on some very basic things.

Happy prompting!

In response to Jeffrey Snover's comment

The above mentioned problems can be further exemplified by the following:

C:\Users> ls


    Directory: Microsoft.PowerShell.Core\FileSystem::C:\Users


Mode                LastWriteTime     Length Name
----                -------------     ------ ----
d----         5/15/2008   1:07 PM            leidegre
d-r--         11/2/2006   1:49 PM            Public
d----          3/1/2008  10:45 PM            sshd_server
d----          3/1/2008   9:35 PM            SvcCOPSSH


C:\Users>

leidegre@monkey-intel /c/Users
$ ls
All Users/   Default/  Default User/  Public/  SvcCOPSSH/  desktop.ini*  leidegre/
sshd_server/

leidegre@monkey-intel /c/Users

The above output illustrates the vast differance in PowerShell and Cygwin running bash. The Directory: followed by a CLR type? I definitively don't need to know when the directory was last modified, nor what attributes are set (if I was I would set a flag for that), moreover the second example is from Cygwin using bash, and it uses colors to show that file actually is folder. The old DOS command "dir /w" used square brackets around the [file name] both are fine mean and lean ways of displaying the appropriate information.

Here is more examples from that horrible output formatting, the pwd command in PS. Yeah I'm well aware that it's a path.

..\ItemTemplates\CSharp\Data> pwd

Path
----
C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\ItemTemplates\CSharp\Data


..\ItemTemplates\CSharp\Data> 

leidegre@monkey-intel /c/Users
$ pwd
/c/Users

leidegre@monkey-intel /c/Users

There is an excessive amount of line breaks taking place almost everywhere, hold back on a few, you don't need to feed the prompt that much. It's nice to not have to scroll a lot within the terminal itself.

I'm using the Windows PowerShell V2 (CTP) and I can't say that it any better with the tab completion.

C:\Users> S<TAB>

into

C:\Users> .\sshd_server

And the obvious problem here is that S would first of all match SvcCOPSSH due to case and secondly is ambiguous. Also note how S is not transformed into .\sshd_server\ which instead necessitates an every so annoying extra slash as sshd_server is a directory. So if I where to cd into leidegre/Documents I'll have to write cd leidegre<SLASH>Documents. On top of this, the second completion will turn leidegre/Documents into cd C:\Users\leidegre\Documents which is longer. Not that big of deal, right? Well, let's look at another example:

..\ItemTemplates\CSharp\Data> cd 1033/D<TAB>

is auto-completed into

..\ItemTemplates\CSharp\Data> cd 'C:\Program Files\Microsoft Visual Studio 9.0\Comm
on7\IDE\ItemTemplates\CSharp\Data\1033\DataSet.zip'

Note that my prompt is different from the default PS prompt in that it only shows the top 3 directories. The full prompt running the defult PS installation is...

C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\ItemTemplates\CSharp\Data> 
cd 'C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\ItemTemplates\CSharp\D
ata\1033\DataSet.zip'

That is quite a lot. Now take into consideration the fact that the terminal most likely is more narrow than you're currently set browser/window and that people more than so will be running several instances of PS. And what about white space in file names? I'd prefer the old fashion programming style of just \  escaping the white space. Mean and lean does have it's benefits. And so, terminals are not mean to be used by just anyone.

bash will suggest the longest match which is not ambiguous, any ambiguity is solved by putting in more input. That's i very important in auto-completion. I'm expected to put in some text, but I will be able to rapidly get exactly what I want.

Things like being able to press CTRL+D or <TAB><TAB> to get an auto-completion list on current input is also welcome. And note that if the list is very long, you'll be prompted that you will be shown +100 items or so.

If you can't unite on any of the above stuff, make it easily configurable. So people can use it however they want.

Best of MIX'08

I love that Microsoft makes all this great content available for free, being unable to attend such a conference for many reasons, it nice to be able to listen to what people have to say without actually being there.
Notably there is one speaker, and one talk that I want to share with everybody.

Developing ASP.NET Applications Using the Model View Controller Pattern
by Scott Hanselman

Now, you might not think that ASP.NET is cool, or that MVC sucks, but this guys is not only hilarious (yeah, a bit off-topic but entertaining), he brings with him a lot more that conventional web development. The presentation runs about 1 hour 15 minutes, and features things from the ASP.NET MVC Framework and a lot of cool neat tricks, e.g. how to use a domain specific language (DSL) for HTML rendering, and more, I'll let you figure it out for yourself.

• http://visitmix.com/2008/
• http://sessions.visitmix.com/ (Page 4)
• http://visitmix.com/blogs/2008sessions/t22/

File decomposition (part 1)

Almost every game share several proprietary file and/or archive formats. When bound by an underlying file system such as FAT, NTFS, HFS Plus or UFS (Mac OS X) you can only do so much, yet some of these come with features well beyond what which you are willing to implement.

I spend a decent amount of time reverse engineering the internals of things, and when it comes to file formats I've looked into a couple of formats that I have found to be more interesting. Among these are MPQ (Mo'PaQ/Blizzard Entertainment), GCF (Half-Life 2/Valve) as well as Scimitar (Assassin's Creed/Ubisoft).

In summary there are a couple things that you would consider when creating a archive/packing format and these are:

Performance, compression and integrity

This is my take on it, and why these things matter.

Performance

On Win32 platforms the WINAPI CreateFile function is not only used to create files but to open them as well. The performance implication of retrieving a single file handle through this function is negligible, but consider a scenario were there is thousands of small files that need to placed in memory. This is an entirely different case were it is much faster to have an in memory representation of the archive structure and do unbuffered sequential block read operations with this single file handle.

The MPQ archive is efficient as it keeps the entire archive structure in memory while maintaining a very small footprint ~64K. If I wanted to open a file in this archive, I would however, need to know the file name, because what I cannot do in a MPQ archive is browse.
It's simple, the MPQ archive uses a hash table to represent paths efficiently. e.g. 'ThisFolder/ThisFile.ext' transform into a 32-bit hash 0x7c14a6c9, which in modulus 32768 turns up 9929. This is a general case, but it means that the path is a file which is described by a block which is found at that index (it's all very efficient in practice).
This is also interesting as in Blizzard games the file is always queried in patch_rt.mpq before any other archive. This gives a way to override any file but it's a solution in which the initial construction failed to address replacement of legacy files within an archive. This became a challenge with the launch of World of Warcraft with massive monthly content updates.

MPQ archives did not initially support archives larger than ~4 GB. Something which became a problem. However, the initial version had almost no room for improvements. Extensions added after the initial release never looked as good as the initial release. A word of advice, if you think you might need it, make room for it. I'm talking about writing for any need you might want later. Mostly reserving binary space in headers. Backwards compatibility is nice, but not really necessary, the reusability is wroth a lot more.

I like the MPQ hash table approach, it's lean and efficient. But I realized while writing this, its a bit hefty to go through it all at once. I will be covering compression and integrity in the future.

I've been looking into the application of ternary search trees (TST) as well as partial match algorithms. To see if there isn't a even better way of creating even more efficient archiving methods.

Quaternions and spatial rotation

Quaternions are non intuitive. A point along the surface of a 4-dimensional sphere, isn’t really something which is imaginable. But if you’re more interested in the application rather than research, well this article might just be for you.

When I started out researching quaternions and their application I was working with animation in games. There's a lot to it, and quite a bit of nonsense too. I found that a bit tiresome so I'll try to get right to it.

The thing about quaternions (I'll refer to them as quats or simply quat, from now on) is that they are defined in a 4-dimensional space referred to as H. This article is about application in 3-dimensions, so one thing you need to understand is that a quat compared to an scalar or vector is never the same thing. One way to look at quats is to visualize them as a rotation around an axis. However, this can lead to incorrect assumptions and wont make much sense in end. The way I chose to picture it, is more like a function with arguments. Input and output in this case is simply something useful in euclidean space (E for short).

A naive camera rotation generally suffers from a gimbal lock, a simple way of avoiding this is by representing the rotation by each axis as two separate quats. Creating these two quats is straight forward, now combine these rotations using the quaternion (Grassmann) product and then extract the rotation matrix from this product.

This was the step that initially confused me. But the trick is to create quats in H space from E values, then do the math (simple product) and finally converting back into E space (extracting the rotation matrix from the quaternion). The result is a rotation matrix which does not suffer from gimbal lock and can transform E space values for you.
In order for the calculations to work you need to use unit quats only.

Vector3 v = {0,0,1}
Q yaw = fromE({0,1,0}, yawAngle) // commonly known as the axis/angle representation
Q pitch = fromE({1,0,0}, pitchAngle)
Q result = mul(yaw, pitch) // magic!
Matrix3x3 m = fromQ(result) // convert back to E space from H space
Vector3 v1 = mul(m, v)

The result is a vector v1, rotated about the two angles yawAngle and pitchAngle around the origin ({0,0,0}). Most people when the first try to rotate things, end up rotating objects around a stick because they fail to realize that rotation is the result of rotation around the origin and translation. If you want the appearance of rotating around some other point you need to translate between that point and the origin before applying the rotation.

There are many great code examples out there and it shouldn't be a problem for you to figure out the precise math involved. Once you get going, it's easy. And hopefully this will help you in getting there faster.

Comments Alongside Code (CAC)

While working on a game I’m currently developing I started to think about how poor some text editors really are. I usually stick with Visual Studio as my preferred environment and it’s a very powerful IDE. But one thing I’m currently missing is the ability to treat any certain range of characters across different lines as a group of text.

Allow me to explain. Back in the old days, any terminal had a fixed number of characters one could display on a single line (typically 72) and basically because of this; you limited any single line to that number of characters. While we still try to avoid writing long lines of code (for readability purposes) we tend to use more characters per line now a days and it’s more a question of what’s suitable rather than what’s readable. I tend to average 80 or 100 characters per line and the readability is still great. But with my screen resolution I can fit up to 148 characters per line and thus wasting about 50 characters, for something which I believe, can be put to better use.

Any modern text editors worth mentioning have a way of doing syntax highlighting (colors to certain keywords or expressions) most text editors also do code folding or outlining which is a way of grouping lines together (this requires some sort of semantic awareness). But I have yet not seen a text editor which can take any rectangular selection as a group of text. I think this is best illustrated by an example.

// A Direct 3D device object other than the one that returned this code caused the 
// hardware adapter to be reset by the OS. Delete all video memory objects (surfaces, 
// textures, state blocks) and call Reset() to return the device to a default state. 
printf( "IDirect3DDevice9::Present D3DERR_DEVICELOST\n" );

One can for instance write comments preceding a certain action which is very common and not wrong in any way at all. But what about writing comments like this:

g_Globals.BeginD3DDReset(); // NOTE to self: this is blocking. 
                            // Blocking the game loop becuase the graphics device
                            // is currently unavailable will totally mess up the
                            // fixed game updates, just ignore the rendering step
                            // until the device is successfully reset.

This is not ideal all the time but this way of writing a comment will allow me to write comments alongside code and I think that rather useful. For this to be really practical, I would expect a text editor to recognize the above comment as comments alongside code (CAC) and conveniently indent the whole thing if any code following the above statement exceed current indentation (see below). It should also adjust the input mode so that these whitespaces used to align the comments alongside code never interfere with the actual writing of additional code. One way to think of this, is to picture the text as a layer on top of your code. This layer, is directly synonymous with the actual text itself and is just a feature by the text editor for positioning comments alongside code.

g_Globals.BeginD3DDReset();             // NOTE to self: this is blocking. 
                                        // Blocking the game loop becuase the graphics device
DoMoreCode( NULL, NULL, NULL, NULL );   // is currently unavailable will totally mess up the
                                        // fixed game updates, just ignore the rendering step
                                        // until the device is successfully reset.

One additional thing which came to mind while writing this is that manual word wrapping is a tedious task. That being said, any implementation of comments alongside code should have the ability to word wrap a comment group through a simple user action. The idea is to augment the experience while still working with text files and never adding any clutter to the actual code. Whitespaces can thus not be considered clutter but storing information about a comment block inside the actual comment would be considered clutter. However, it would also be useful as it could provide sensible metadata about the comment itself. E.g. one could chose to treat a comment beginning with a certain word or pattern as a certain marker.

I think it’s obvious at this point why this would be a very useful and powerful feature. Not only as an extension but it could theoretically provide a lot more functionality than just comments alongside code when working with plain text files.

Hopefully I will be able to revisit this topic with a working demo sometime in the future but I leave no such promises.

To all you Emacs and VIM users, if you think you got it, then you're wrong. I use VIM myself and VIM does not do what I want to do. You can do rectangular text selection, but the editor needs to recognize the comments as more than just comments. It a block which needs to adjust to it's surroundings.

Game loops – Not to be taken lightly

Not too far back, I started developing games. Back then I decided to hit the books for a while before getting upfront with the actual programming and one of my questions was about game loops. While googling the web I came across this article (a must read for anyone not familiar with different game loops) and must say that I was a bit surprised how a rather simple thing as a loop turned into something a bit more complex, but for the better!

I must begin to thank Koen Witters for his post about game loops as it helped me a lot. However, it still took me some time to wrap my head around the constant game speed independent of variable FPS loop, and decided to write this in order to work out some of the questions I had.

This is a modified version of the suggested game loop by Koen Witters:

typedef T_TIMEINFO struct {
    int updates_per_second;
    int frequency;
    int skip_ticks;
    int max_frameskip;
    int next_game_tick;
    int loops;
    float interpolation;
} TIMEINFO;

TIMEINFO t;
ZeroMemory( &t, sizeof( TIMEINFO ) );

t.updates_per_second = 25;
t.frequency          = get_frequency();
t.skip_ticks         = t.frequency / t.updates_per_second;
t.max_frameskip      = 5;
t.next_game_tick     = get_time();

while( true ) 
{
    t.loops = 0;

    while( get_time() > t.next_game_tick && 
        t.loops < t.max_frameskip ) 
    {
        update_game();

        t.next_game_tick += t.skip_ticks;
        t.loops++;
    }

    t.interpolation = float( get_time() + t.skip_ticks – 
        t.next_game_tick ) / float( t.skip_ticks );

    update_render( t.interpolation );
}

The variables here have been moved into a struct to make them more easily managed and accessible. Also, time is treated as an incremental value and only through the frequency can you find a relation to actual seconds. This is important since high-resolution timers come with a variable frequency (Hz) while the GetTickCount() function returns a discreet number of milliseconds (1000 Hz).

So without further due I present to you this diagram of the above running game loop:

Let’s start with the scheduling as it’s essential to any game loop.

Each game_update (red dot) occur after deadline (black dot) and thus there is always a difference between game_update and deadline denoted here by overdue. This relatively small value can be used in various ways e.g. in a multiplayer game to check that game state is synchronized among participants, as the overdue value is simply a metric of how far game_update is behind. Note that this value is only accumulated if game_update requires more time than skip_ticks and if so, another game_update is scheduled immediately after.

I bet you went – “Eh? What the… the timer is all backwards!?” when you first looked at the above code. It sure isn’t trivial but it’s essential. You have to think about it as two separate time lines; one continuous and one sampled. While we can plan ahead exact moments in time it’s unlikely that we’ll be that punctuational. If we’d assume we’re always late and plan our next deadline always skip_ticks after last deadline we not only prevent drifting, we also make up for any subtle interrupt game_update might cause. In practice we will either schedule game_update a little before or little after actual skip_ticks but as long as this overdue error remains small, it’s perfectly fine.

It’s possible that game_update might stall (be interrupted) and take very long to finish due to some unexpected behavior. When this occurs the next deadline might be scheduled within the overdue error and immediately run game_update until either caught up with real time or max_frameskip force game_render. A max_frameskip is good, it will give the impression that your game is actually still running, slow, but running. However, getting behind and accumulating an overdue is very bad and you should be aware of problems that might arise from this.

And that’s it! Now all the remaining time between each game_update is spent on game_render. The game_render have to implement prediction and interpolation if you want to smooth results. This is fairly simple and Witters covers that which you need to know in his article. This interpolation value is conveniently always between 0.0 and 1.0 depending on the intermediate step of the current iteration. So the only thing you need to be concern about is how to represent a progression between each fixed game_update.

Ad free Windows Live Messenger

It's actually very simple. Windows Live Messenger get's all it's ad information from a host called rad.msn.com (remote advertisement?) and since it's possible to tell Windows to look up invalid IP addresses through the host file, one can quickly get rid of the ads in Messenger.

You need administrative privileges for this, and if you are running Vista with UAC enabled you need to make sure you launch both Notepad.exe and Cmd.exe with elevation (Shift right-click them in the start-menu and choose run as administrator).

• First, find you host file, Microsoft Windows (NT/2000/XP/2003/Vista) is located in %SystemRoot%\system32\drivers\etc by default.
• Second, open Notepad.exe as administrator (and with elevation if you use Vista and UAC) choose File > Open... and navigate to the host file (make sure you show all files) and open it. It's called "hosts" without a file extension.
• Now add the following new line at the end of the file: "0.0.0.0 rad.msn.com". Save the file and close Notepad.

It will tell Messenger to look up an invalid IP address for ads, and thus display no ads. You need to flush the DNS cache for this to take immediate effect, and it can be done by opening the command prompt Cmd.exe (as administrator) and type "ipconfig /flushdns". If you wait long enough Messenger should stop displaying ads but the quickest path to salvation is restarting Messenger as well.

Enjoy your Ad free Windows Live Messenger.