[freeside-devel] Installation with legacy data

Dave Burgess burgess at neonramp.com
Mon Dec 3 13:05:17 PST 2001


ivan wrote:

> For what it's worth, I don't import legacy data directly into the
> database; I use the Perl API.  There's a layer of sanity-checking that
> you're bypassing.
>
> See bin/svc_acct.import, bin/svc_domain.import etc. as well as
> eg/TEMPLATE_cust_main.import
>

I tried, I swear I did.  I'm reticent to complain, since I haven't taken the
time to provide a solution, but I spent a month trying to get the old data
loaded with the import routines.

There are two problems with the code.  The first (and least of my problems) is
that the sample code you provided in the base system is a little opaque and
uses some assumptions that didn't work well for me.  For example:

  - I use the Unix 'expire' field to turn access to the servers, and turn mail
on and off: this allows me to 'pre-disable' a customer or establish defaults
for classes of users.  I've modified all of my systems to honor the expire
date, so it works for me.
  - I use the 'Group' check item in my RADIUS database to decide who gets on.
I also use it to decide control the 'types' of users,
dial-up/DSL/ISDN/mail-only/ftp-only/etc through the use of the 'group' RADIUS
check item.  The current import code assumes that the relationship between
RADIUS and /etc/master.passwd is 1:1.
  - The current RADIUS import code only imports RADIUS v.1 config files.  The
Version 2 config is close, but not exactly the same.

Like I said, I'm not going to complain about the import code since I'm not
willing to do the hard work of getting the import program to actually do what
I need it to do.  Instead I did the hard work of putting in as much data as I
could by hand and thenfixing the database through the GUI.

The second is that that data that I was importing is in a badly designed 1NF
database written in Access about 5 years ago.  In other words, unless there
was a natural language interface that could figure out things like "Cust Ref
Bill at Autocon/dial/dsl OK from 13 to 27/nc 'herman' through Aug" in the
comments block, there was little that it could do for me.  Getting the
password file loaded was a snap; getting the customer data loaded was a
breeze.  The payment history stuff was a little odd, but I got it working.
Everything else was cobbled together with bailing wire and gum.

What I should do now is write a 'sanity checker' for the data that I have now
and see if I can't get the rest loaded.  We have already connected 3000
services to about 500 of the various billing records.  Of course, there are
subtle little problems along the way, but I'm learning the internals of the
system well enough that I can solve most of them.

I've also gotten really good at building svc_www records from Microsoft
Access.  I can so it from the MySQL interface too, but it's more fun the easy
way. :-)

I'm planning a MySQL Ver 4 upgrade to see if these new sub-selected selects
will work.  I've already figured out (using inner joins) ways to do a few of
the new selects, but a couple are just too subtle for me.

I'm on the case, though - 'viva freeside'.

Dave

> On Tue, Nov 27, 2001 at 01:26:12PM -0600, Dave Burgess wrote:
> > I'm putting together the first draft of my "Importing legacy data into a
> > Freeside Database for Idiots" document.  It is currently at
> > http://www.cynjut.net/pub/freeside.imp.txt for anyone that would like to
> > see it.
> >
> > It's a work in progress (as is the database implementation) so anything
> > you find that is incorrect, please let me know.
> >
>
> --
> _ivan




More information about the freeside-devel mailing list