[impdev] Redmine issues to SourceForge tickets migration test.

[David Seikel]

Getting Redmine to work on SourceForge's servers is very problematic.
The SF support people told me that no one has managed to do so yet.  I
considered that a challenge, I like challenges.  While I did manage to
get it working, it's really very slow.  Slower than our existing
version, which itself is too slow, and times out a lot.  So, I worked
on Plan B this weekend.

I've written a script to migrate from Redmine to SourceForge's Allura
ticket system.  For all of he things we are running servers on SF's
systems for, we plan to migrate them to Allura at a later stage, so
this is not so much Plan B, as jumping the gun.

I did a test run tonight, which took about six hours.  Could you all
please take a look at the results at
https://sourceforge.net/p/team-purple/imprudence/tickets/ and
https://sourceforge.net/p/team-purple/kokua/tickets/ and let me know
what you think?  Keep in mind this is only a test run, there are known
issues, and there might be some changes that you suggest.  There will
likely be another test run, but later I'll wipe this all out and do a
real run.  Please also comment on the issues below if you have any good
ideas.


Known issues (some of these will be solved) -

Only the Imprudence and Kokua issues are migrated, not the various web
site ones, which only have 52 issues between them, all of which are
likely obsolete after this migration anyway.

It's not possible to faithfully recreate tickets, sometimes due to
security issues.  For reference there is a field at the top
"redmine ID" that is the issue ID on the original Redmine server.  We
can probably have a copy of our Redmine running in read only mode on SF,
and have this field link to it.

All these tickets are created by team-purple-bot, thus the ticket
creator is not listed as who ever really created it originally.  This
is why there is an "original creator" field at the top.  The same goes
for creation time, the Allura creation time is set to when ever the bot
created the ticket.  At the top of each ticket I inserted text stating
the original creation time, creator, and a link to the original Redmine
issue.  A last comment is added that mentions when the issue was
migrated and other details.  Each post has a similar line of text at
the top with original details.  This might be overkill, so perhaps some
of this can go away?

There is several hundred users on our Redmine system.  In some cases
Redmine only provides their user ID, and I only translate that to a few
of the users, mostly those that are actually listed on the project
overview pages on Redmine.  For those where I know their SF account
name, I link to that.  There's more places where I could add those user
links.  I could do a look up for those other names to.  If we keep a
read only copy of Redmine on SF, we could link to every ones user page
there.

I didn't bother to translate Redmine markup to SF markup.  It was
rarely used, and in those cases where it's important, one of us can fix
it up anyway.  There is enough similarity in the two markups that some
worked anyway.  However, 
https://sourceforge.net/p/team-purple/imprudence/tickets/410/ shows that
it's not consistent about what works.  Redmine uses "#123" markup to
refer to other issues, SF uses "[#123]" the same way, but I didn't
translate them either.

Since the uploaded files from Redmine are already on the SF project web
space, I just linked to them rather than figure out how to deal with
the undocumented Allura API for uploading files.  The files themselves
include a twelve digit timestamp at the start of the file name that was
added by Redmine.  Redmine has a download wrapper that strips that off,
but we don't have that luxury.  I could probably write another script
to strip those timestamps off on the file system, but then we have the
problem of how to deal with duplicate file names.  Meh, think we can
just live with the timestamp.

The relationships between issues had to be dealt with in two passes,
since if an issue relates to a later one, then that later one has not
yet been created in SF, and thus the script wont know the SF ticket
number to link to.  So it collects the details during the first pass,
then updates SF tickets in the second pass.  Due to this being a first
test run, which crashed a few times, only the last 150 or so have
properly been linked.  Hopefully the real run wont crash and all
related issues will be linked properly.

The categories could do with some clean up.  The script is structured
in a way that it would be easy to massage these to new ones, some of
that is done already.  For example Imprudence uses "UI", Kokua uses
"User Interface", the Kokua method is a bit more user friendly for
noobie issue reporters.

Due to a limitation in the way SF handles status, they where renamed a
little.

SF has no concept of custom large text fields.  So "affected versions",
"repro steps", and "system info" where tacked onto the original
description.  Related issues are also dealt with in the same way, since
SF has nothing that could deal with that.  The new SF ticket creation
method includes a big help blob at the top suggesting people add repro
steps and system info to their original descriptions.

SF has "Milestones" hard coded into it, "Target version" was converted
to that.  Can't change the label SF supplies.  Kokua doesn't use them
anyway, but Impy does.

I could not get updating the old Redmine issues to work.  Jacek
suggested adding a comment on them pointing to the migrated version, so
that people that are watching issues know to go watch the new one
instead.  I think it's an authentication issue, perhaps Jacek could
enable the API key stuff in Redmine?  This can be done in a separate
pass, so it's not a blocker.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.imprudenceviewer.org/pipermail/impdev-imprudenceviewer.org/attachments/20140212/a7448366/attachment.pgp>