VSS to Subversion: What to migrate?

In what amounts to my first round of general musings about migrating a Microsoft Visual SourceSafe repository into Subversion, its clear that one critical decision affects the entire process: What will you migrate?

The question sounds simple on the surface, but once you begin to peel away the avacado layers from both Subversion and SourceSafe, you find that the differences in the storage philosophies of either make that decision much more critical up front.

The simplest type of migration is to "Get Latest" to snag the contents of the latest revision of each SourceSafe project, create a Subversion folder to host the migrated copies, and perform an "svn add" to pull the files into Subversion. That's the clear choice if you're not worried about migrating any of SourceSafe's history and want to create a new source control start point with Subversion. It's also a migration you can accomplish in a day or less. If you want that history, however.....

Migrating SourceSafe's history requires delving much more deeply into the way it stores and tracks changes, and reconciling that against how Subversion does the same thing. Doing that reveals more of the inherent fundamental differences in the way the two tools operate, and gives you a first-level notion of how hard it is to get that history into Subversion.

Subversion stores sets of changes to multiple files and/or folders as transacted sets of changes. Each set of changes is assigned a revision number. SourceSafe tracks changes on a per file basis, storing the entire revision of each file in each change. That means that each and every change could be migrated as a single revision into Subversion, making for a huge repository that is 99.9% changes, or that those changes must be aggregated in some way that bears no relationship to any native structure from SourceSafe.

A nice open-source migration tool, VSSMigrate, tries mightily to do that for you. We'll talk about that next time.

MIgrating VSS to Subversion

If you're a long-time Microsoft development shop, chances are you've had to deal with its source control tool, Visual Source Safe. After years of wrangling with VSS and dodging the bullets of corruption and loss that have formed its reputation over the years, I'm undertaking the task of migrating our VSS repository to Subversion. Yeah, I evaluated Team Foundation Server, but it was just too much.

Don't know how many out there might be contemplating the same task, but I'll be posting a few musings here and there about my progress, obstacles, tools, and lessons learned along the way. Hopefully, something I encounter will help someone else.

For starters, the Subversion repository will be entirely Windows-hosted - unfortunately, slapping Subversion on a Linux server (which is my preferred option) just isn't going to work. Also, I'll be installing the TortoiseSVN repository client and the AnkhSVN source control plug in for Visual Studio. And I've already taken quite a few steps in migrating the repository with the C#-based open-source tool VSSMigrate, which is hosted on CodePlex, and uses the SharpSVN .NET-based Subversion API. Considering that I'll have an objective of migrating just about everything in VSS, including deleted (but not purged) items, I suspect the process even with VSSMigrate will be tedious.

As I post my progress, feel free to post any questions or thoughts you may have as I Go Forth and Migrate - and hopefully wean away from SourceSafe!

SVN+GSSAPI : "Could not obtain list of SASL mechanisms" solved..kinda?

It doesn't take much of a Google query to find a good chunk of users trying to solve a seemingly intractable problem with their Subversion setups trying to authenticate through SASL via the GSSAPI, often back to a Windows Kerberos server.

This search stems from a chronic receipt of the title message:

"Could not obtain list of SASL mechanisms." with no error logging, and a seemingly correct configuration among the various configuration files for subversion - notably svn.conf and svnserve.conf.

For background, Subversion users can tell their SVNSERVE process to authenticate via the external SASL library. SASL, in turn, supports a variety of authentication mechanisms, including a local database, LDAP, and Kerberos. Set up the SVNSERVE.conf to enable SASL, and then write an SVN.CONF file that defines the proper plugins and authentication mechanisms, and you're off to the races. In most cases, this sequence works fairly well.

I needed to have SASL authenticate against my Windows AD/Kerberos server, which was (supposedly) possible by employing the GSSAPI mechanism in my svn.conf file, along with a keytab to identify SVN to AD. But blast if I could ever come close to making it work - with the message above my constant response.

I was plodding down the same road until I noticed amid a zillion lines of ProcMon traces and NetMon captures one interesting note - that the keytab file I had created that held the SPN to authenticate to the Windows Kerberos server was never being read. Procmon showed that the configuration file, svn.conf, was being read, and I knew the keytab designation was there. Yet it was never touched. That made me wonder what was going on....then, after what seemed to be the umpteenth Google search and iteration, I found this little gem arising from a Gentoo forum in which some poor soul was translating the same error from a German server (a link which you can follow here. and discovered this:

From what I gathered, it seems as if it might have been possible to use plain svnserve+sasl+gssapi two years ago. While the same text I linked to in the previous post is still in the source tarball, a little grepping and searching of bugtrackers revealed, that the files needed to do so are no longer in the source.

Bottom line? SVN + GSSAPI + SASL may have worked at one point in Subversion, but apparently not anymore. 

Wow.

Open source software can be a great thing. Working several hours to make something work, only to find it isn't possible, isn't among them. This post is for anyone trying to resolve the same problem.

Caveat open-source emptor.

HOW-TO: Implement single-file externals in Subversion 1.7

Over the last few days, I've managed to become at least competent at a few of the basics of Subversion. One particular issue that had eluded me was the implementation of file-level externals (svn:externals), which was new to Subversion starting with version 1.6. 

Search after search of blog post and commentary about how externals were implemented led me on a frustrating and often circular chase through the same set of mirrored references, and then back to the Subversion online manual. Finally, however, after putting together some pieces of information about how directory externals were implemented, I broke the code for file-level externals.

While most others may have mastered externals at the file level, I'm still a noob, and suspect others may be in the same boat. As a result, even if for no other reason than my own reference, I'm going to toss out my own spin on how to implement file-level externals in Subversion. This is not a ground-up discussion of Subversion - there are plenty of places that offer such information.

Basics: This implementation was tested on a Subversion 1.7.4 setup hosted on a Slackware Linux server, and accessed via TortoiseSVN 1.7.6, Build 22632, 64-bit version under Windows 7.

First, some background. 

Subversion as far back as version 1.5 supported the notion of external directories in a repository, meaning that a directory could actually deliver files from a different location when checked out. That concept was extended to files in version 1.7.

External directories are configured by applying the "svn:externals" property to the directory, and supplying as the value of that property a list of folder/URL pairs that represented local folders that were actually redirections to folders specified in the corresponding URL.

Let's say you have a project with source that will be common to multiple projects, called "Shared", and within that project is a directory called "SourceFiles." Taking a page from Subversion's documentation, suppose their "calc" project needed to avail itself of that SourceFiles folder at its trunk. To accomplish this, change to the local working copy of the calc project (c:\calc\ for our purposes here), and type

svn propset svn:externals "SourceFiles svn://server/Shared/SourceFiles" trunk

property 'svn:externals' set on 'trunk'

Running a simple "svn up" now pulls down the specified external directory.

It turns out that External files are configured by applying the same "svn:externals" property to the directory, but supplying as the value of that property a list of file/URL pairs. What confused me mightily was the Subversion 1.7 docs that said file externals were implemented the same way directories were implemented, which led me to believe you needed to create dummy files to which svn:external properties were added. What they really meant to say is that such external references were declared at the directory level just as external directories were. 

So, in practice, suppose in the above scenario you wanted to reference only the SharedClass.cs folder into calc/trunk from SourceFiles rather than the entire directory. The trunk folder still receives the "svn:externals" property, but this time gets a file alias rather than a directory alias:

svn propset svn:externals "SharedClass.cs svn://server/Shared/SourceFiles/SharedClass.cs" trunk

That, followed by

svn up

gets

Updating '.':

Fetching external item into 'SharedClass.cs':

A   SharedClass.cs

Updated external revision to XX 

And there you have a simple external file logically assigned to one project, but physically residing in another!

I hope this helps any other SVN newcomers out there trying to accomplish the same goal!