[freeside] Page refreshing

[Jason Spence]

On Sun, Dec 03, 2000 at 02:13:11AM -0800, ivan developed
a new theory of relativity and: 

> Hmm... interesting.  How does that compare to "Pragma: no-cache" which is
> added by CGI.pm?  (look arount line 1261 or so).  Certainly can't hurt to
> add it if helps.

I believe that the cache-control stuff is a different mechanism from
the expired stuff that Freeside (well, our internal fork of Freeside)
does now, if I remember correctly.  I haven't synced up to the real
Freeside in almost two months now, so I don't know what the headers
look like currently.

> > BorderManager firewall/NAT box access Freeside, since BorderManager
> > has some weird cache which (as far as I can tell) turns on randomly
> > and causes all sorts of havoc with Freeside.
> > 
> > We're going to install squid at the site we've deployed Freeside at
> > soon, so I think it's time that Freeside learned how to deal with
> > caches.
> 
> I'm pretty sure Freeside deals okay with well-behaved caches.  I run squid
> myself.

BorderManager is certainly not a well-behaved cache, especially
considering that our sysadmin is Really Sure that the cache isn't even
enabled on at our site.  *sigh* I tried a couple things that you guys
mentioned and the problem seems to pop up less often, which makes me
even more suspicious that it's a caching issue.

> >  I'd write a patch, but I already have (I think) three ones
> > floating in Ivan's queue and I haven't received any response on those
> > yet, so...
> 
> I know, I'm really behind, I'm sorry and I'll do my best to catch up.
> 
> Also, are the folks with pending patches (that would be you, Jason, and
> also Kristian and Jeff F.?) CVS capable?  I'll still be a bit of a
> dictator and ask that the big changes be checked in as branches, not on
> the trunk right away, but no reason for you folks not to be able to check
> in bugfix-type stuff on the trunk.  And stuff on branches is a lot easier
> to work with than patches.  :)  If you're amenable to using CVS, let me
> know and I'll setup accounts. 

Yeah, we're using CVS to maintain our version of Freeside.  We're
doing it on a separate branch, but I'm not very clear on the procedure
to check that branch into your trunk.  Could you please lend me a hand
with this?  I vaguely remember something about two -r flags and -kk,
or maybe this cold is affecting me more than I think.  *sniff*

I've used CVS to do simple commits to the trunk on other open source
projects before, so I won't have any problem there.  Would I have to
do my Javascript stuff on a separate branch, or the trunk?
 
> > Kristian, it sounds like you've already tried this.  Do you have a
> > patch for the CGI.pm file that changes the headers?  
> 
> Hmm, it does seem like CGI.pm is the right place for it, but even if it
> got changed today, CGI.pm has been distrbuted with Perl for ages.  Have to
> live with it for now.  Until then:
> 
>     package FS::CGIwrapper;
>     use vars qw(@ISA);
>     use CGI;
>     @ISA = qw(CGI);
>     sub header { shift->SUPER::HEADER( @_, '-Cache-Control' => 'No-Cache' );
> 
> then s/use CGI;/use FS::CGIwrapper/ and s/new CGI/new FS::CGIwrapper/; in
> htdocs/*
> 
> should do it.  I checked in a (slightly cleaner) CGIwrapper.pm, but didn't
> yet make the changes in htdocs/ - if one of you guys wants to test it out
> before I do that (or you do it, heheh!) that would be great.

Aw man, now I really do have to sync up with your CVS.  Well, maybe
not; that looks like a pretty easy thing to change in our branch.
Actually, that's close to trivial.  Nevermind.

One of the things I've been playing with has been a system that takes
generic data validation rules and automatically generates both the
perl and the javascript code necessary to do the data validation.
That way, you write out a string like "##########" and you do 

==============================================================================

Fry's Electronics Feedback Form
</TITLE>

<SCRIPT language="perl">
$v = FEBO::CGI::Validator->new();

$v->set_rule("##########");
print $v->get_js();

</SCRIPT>
</HEAD>

... time passes...

else { /* process input */
  $v->check($cgi->param('daytime_phone_number'));
	if(! $v->valid()) {
      $error = "Invalid daytime telephone number specified.";
    }

...
}

==============================================================================

I think it's a very nice way to guarantee data integrity and avoid 90%
of the double-coding you have to do when checking data during CGI
processing as well as with client-side javascript.  Is anyone else
interested in this system?  I'm trying to figure out if this should
really be a part of the already huge CGI.pm or if I should package
this with Freeside or make it a separate perl module or what.  

> > Actually, I just had a thought: the "browse customers by customer
> > number" page is pretty beefy,
> 
> Oh, not worth it; the idea is wrong for a large install.  Browsers just
> have problems with large documents like that.

Actually, we've been using it as a stress test for Linux browsers :)
We're up to 800 customers in our for-real Freeside installation.  The
nightly mozillas seem to handle it great.  

> For a large install, I'd say just skip having that as an option at all -
> it'll just suck up absurd amounts of system resources.  Have people use
> the fuzzy search.  It'll suck resources too, but it won't crash browsers
> as much.  :)

Heh; we're running it on "absurd amounts of system resources", so we
don't notice :)  The sales people kind of need it, see below.

Should be pretty easy to have the system stick a message in the log
saying that you're doing something silly...

use FS::cust_main;

@c = qsearch 'cust_main', {};
if(scalar(@c) > 1000) {
  warn "Using browse customers is probably not a good idea..."
}
if(scalar(@c) > 10000) {
  $error = "Too many customers in database; disabling browse customers report.  Please use the search feature instead.";
}

... and so forth.  

The main complaint that I had from the sales people was that there
wasn't a real good way to do certain types of searches (I'm picturing
something like bugzilla's flexible search form).  I wrote a search
form by customer ID, but they keep confusing the customer number with
the package number.  ... oh wait, duh, I'll just increase the font
size of the customer number and add it to package view as well.

...

Ok.  The other problem is that sometimes we have to lookup the last 20
customers that signed up, and I haven't written any code to deal with
that situation yet.  Any recommendations?

Oh; something else: I've been messing with Text::Template and
signup.cgi.  Ivan, we've discussed the different templating solutions,
but I never heard you say that Freeside is going to use so-and-so
templating system.  We need one soon, because I just can't deal
with the change requests that I get for the signup.cgi page.  Have you
decided on a templating solution yet?  If not, I'll just go ahead and
use Text::Template for this case.

 - Jason