Partition for speed

• Application partitioning

  • stop and think - work smarter not harder
• Pick the right database for the job, if you have the choice
• Work close to the data
• Use stored procedures where appropriate
• Moving data to/from the client is always expensive
  • think of latency as well as bandwidth
• Use proprietary bulk-load where appropriate
• Consider database maths vs. Perl maths
• Multiple simple queries with 'joins' in Perl may be faster
• Consider local caching, in memory or DBM file etc.
• Mix 'n Match techniques as needed

Previous slide

Next slide

Back to first slide

View graphic version

Database logical design and physical implementation
are always first priority, but beyond our scope here.

Application partitioning

Work close to the data - actually inside the database is best

Stored procedures
many benefits where they are applicable

language often painfully limited compared with Perl

Moving data is expensive -bandwidth (bulk) and latency (iterations)

Bulk-load - use proprietary tools for maximum speed
- you can use perl to massage the data into the right form

- bulk-load tools often have limited functionality
use DBI for maximum flexibility, such as update-else-insert