Archive for the ‘frugal innovation’ Category

The Case of a Platform Rebuild from Laravel

This post is a discussion with a colleague who reached out to me requesting for advice on whether to rebuild their successful e-commerce platform whose usage has grown exponentially over the last 18 months.

Advice on Platform Rebuild
Laravel is not Enterprise Ready

My first piece was 90% of platform rebuilds and re-architectures fail, especially since there are always unseen constraints in the new tooling that slows down the process. My approach is always to advice to squeeze as much as you can from your existing framework, language and platform before starting to look elsewhere

While this post is Laravel – specific, the ideas are applicable to any framework, programming language or even mix of technologies

As a Laravel developer who is passionate about architecture and solving scaling challenges, this is a series of questions to drive decision making

  1. Are you on the latest PHP 8.1 and Laravel 9?
    • Uprading to the latest version of PHP and Laravel pulls in the latest performance improvements in the language stack
    • This also applies to the webserver Apache/Nginx
    • Is your database the latest applicable version
    • Server hardware
      • CPU latest generation of processors for the configuration
      • OS patches – remove unused and unneeded services
      • SSD disks for IO bound performance
      • RAM – the more the better
    • Network configuration – unnecessary round trips from the application to the database (see below too)
  2. Is your database and application well tuned? Indexes, reducing the size of requests and number of queries
    • Application tuning is an art that can be taught – for a database heavy application like this case the Eloquent Performance Patterns by is a key resource (https://eloquent-course.reinink.ca/)
    • This also involves removing unused plugins and libraries, and leveraging the framework best practices – Larvel Beyond Crud (https://spatie.be/products/laravel-beyond-crud) and Frontline PHP (https://spatie.be/products/front-line-php) from the team at Spatie are excellent primers
    • The memory usage per request can easily be shown by the Laravel Debug Bar (https://github.com/barryvdh/laravel-debugbar) during development
    • Checking the size of requests and responses may improve performance – send as little data as required for the operation, paginate in the database not the application
    • Add indexes to improve table joins, redesign tables for needs, separate data collection & reporting (denormalization may be needed here)
    • Use Specific API requests for cases where generic default CRUD ones do not perform well
    • Is all that Javascript necessary in the application pages?
    • Is all that that CSS necessary in the application pages?
    • Is the database tuned for the hardware that it is running on?
  3. Are you using the framework tools like queues to offload processing into asynchronous operations via queues
    • At times performance issues may be experienced by users especially in e-commerce setups where actions that can be asynchronous are added into a synchronous workflow slowing down responses to end users e.g., sending a confirmation email (adds 2-5 seconds to the order response) can be done after so the ordering process completes faster for the customer
    • For faster throughput using extensions like Octane can provide the necessary performance boost
    • Tweaking the web server Apache/Nginx to handle more users
    • Load balancing across more than one webserver can provide quick wins
  4. Increasing your database resources – bigger server, tuning the MySQL
    • Computing power is cheap thus a beefier server or more RAM may do the trick depending on whether the deadlocks are CPU/Disk or memory bound
    • If a beefier server does not help, probably load balancing across multiple smaller servers even for stateful vs stateless requests could improve performance
  5. Concurrency using Laravel Vapor and Octane?
    • Vapor is a paid service to bring serverless to the Laravel applications
    • Octane increases the concurrency of request handling though application changes may be needed to cater for the constraints Octane places
  6. Are you profiling your application with tools like Blackfire or Sentry to find performance deadlocks?
    • The performance improvement approach is measure, find the bottleneck, tweak to improve, then rinse and repeat
    • Are there errors or failures that are causing the application to slow down
  7. Have you refactored and cleaned up your data model to match the current reality?
    • As applications evolve there is a need to remove old code, data columns to suit the new reality
    • Refactoring code to match reality removes any unncessary baggage that is carried along
  8. Is your architecture the simplest that it can be – https://future.a16z.com/software-development-building-for-99-developers/ as your organization may not need the complexity of a Fortune 500 or FAANG
  9. There are hidden gems which can also be leveraged to improve your architecture & unearth performance issues
    • Test Driven Development – even if tests are written after the code, unit testing complex algorithms while carrying out end-to-end workflows of critical user paths
    • Design for failure – especially for external services
    • CI/CD – automate deployments to get new features, bug fixes and enhancements into production as fast as possible
    • Setup staging sandboxes to test out ideas and tweaks
    • Monitor your production application service health – Laravel Health (https://spatie.be/docs/laravel-health/v1/introduction) provides an excellent starting poing

When all else fails and a platform rebuild is necessary the Strangler Fig Pattern is a great way forward – replace different parts of the application as needed replacing them with newer architectural pieces referred Legacy Application Strangulation : Case Studies (https://paulhammant.com/2013/07/14/legacy-application-strangulation-case-studies/)and Strangler Fig Application (https://martinfowler.com/bliki/StranglerFigApplication.html)

What are you thoughts and suggestions? What has worked for you and what pit falls did you find? Any additional advice?

Tech Tip: Whatsapp Groups for Record Keeping Purposes

Over the years my wife and I have been sharing lots of information via Whatsapp some of which we need for tracking purposes, the most common being weekly metrics from a business operations where I am a silent partner but key stakeholder

One neat trick that I discovered from a friend, I have forgotten who, was to leverage Whatsapp groups as a record keeping tool, this group only has the 2 of us, is named Farm Finances, and only contains the information we need to share with regard to the specific topic

Try this for your next business endeavor, wedding, construction activities, long or short running project whose records you do not want to have across the noise of day-to-day communications as you prepare to move this into a more formal tool

As I wrote this I figured out that you can also do the same for keeping notes for yourself, to-do-lists, project reminders etc

Perspective: User Requirements for Technology Projects

I was asked to talk about the handling of user requirements, how to link them to implementation within technology projects, by The Medical Concierge Group (TMCG) a digital ehealth service provider.

The key principles are being able to respond to change (agile), keep learning, and how to capture business/customer outcomes as well as improving communication across different departments and external stakeholders.

A Frugal Innovation Approach to Simcard Verification 2019 Edition – Design & High-level Architecture

Happy Easter Monday to you all, and I hope that the rains on Sunday night completed the cleansing process from the festivities, the resurrection of the Lord Jesus Christ and from the feast of the Goddess Ester (depending on which side you lean)… I am one who embraces all religious doctrines an faiths.

So over the last few days I have started receiving a message from MTN Uganda, to physcially visit a service center to verify my sim card registration, well this is only the 3rd cycle for selected customers whose details were screwed up during cycle 2, and I guess I am one of the lucky few with time to waste.

This is a followup to the 2013 recommendations for Simcard Registration https://ssmusoke.com/2013/03/12/uganda-simcard-registration-alternate-approach/ which apparently were not providing sufficient value 😉

Anyway after having to make 10 calls this morning, the reminder message, a hardcoded IVR message, has left me frustrated, but also wondering, why do I have to physically visit the service center, it is 2019!

Rather than complain all the time, I focused my anger with support from my trusted colleagues at Styx Technology Group (http://styxte.ch) we got to protoyping a quick and dirty solution to this mess. What MTN and the regulator need are my National ID details, since they will scan the ID or take a photo of it, take a photo of me then I will have to wait 2-3 days,

A frugal innovation can be:

  1. Mobile App front end to capture data that is needed
  2. A backend system – doesn’t matter what it is – can even reuse the exisiting simcard registration database they have with processes to complete the verification flow, and link into the audit process that triggered this verification
  3. A verification process, which can be done by the app automatically, or using a backup USSD channel. This follows 2FA (two factor authentication to prevent mis-use)
  4. A notification that the verificatio process has been completed and *197# can be leveraged to check status.

This method is not for everyone, but provides a solution for those of us who may not be able to line up and waste 2-3 hours in line to do just this…

Some mobile screenshots from the design team

 

NIN Details

National ID Details

Phone Numbers

Phone Numbers

Confirmation

Confirmation

Thoughts and additions are welcome!!!

UPDATE 1: One of the team members asked me, so does this solve your problem? How do you know which numbers are listed on your NIN that has been provided? Leading to iteration 2 of the Phone Numbers screen allowing the display of existing numbers with functionality to remove currently registered numbers…

Phone Numbers - v2

Syzygy – Release 2 – Uganda Mobile Money Cost Estimator

Syzygy is a Uganda focused utility calculator (launched by this post https://ssmusoke.com/2015/08/11/launching-syzygy-uganda-focused-utility-calculator/), this new release adds a tool not available anywhere else allowing you to estimate the costs of a mobile money transaction. Are you sending to a registered number, unregistered or trying to withdraw from an agent or ATM?

Hope this app saves you the hassle of searching for the transaction chart which is usually hanging in the agent’s booth.

Do let us know what you think of the new release.

The link to the app is http://bit.ly/syzygy-ug

A screenshot of the new calculator is below

Mobile Money Cost Estimator

Mobile Money Cost Estimator

%d bloggers like this: