Archive for the ‘devops’ Category

5 Jan

Automating the Deployment and Scheduled Execution of an R script on a VPS using Laravel Forge

Posted by ssmusoke in ci/cd, devops, tips and tricks. Tagged: .

I love data, its in my blood, I love analysing, cleaning it and preparing it for others to use, still have to get to the last mile to displaying and visualizing it but that is a story for another day

2024 for me is the year of “automation”, for everything that I do. I have been polishing up my R skills over the holidays, as a way for moving into Python based data wrangling, however the challenges that I have faced is to do with getting my R scripts to run regularly – this has to be on a server connected to the internet since my laptop is not always on

I tried ShinyApps in the past, but this needed me to learn more things which I did not have bandwidth for.

So when Contabo my host wiped my dev VPS during the holidays, I had to re-build my sites, one of which was a documentation site built using PHP and markdown, which I wrote about Deploying a Tighten Jigsaw static site to a VPS using Laravel Forge I was deploying generated HTML to the VPS, then it hit me can’t I do the same with an R script

Setup and Configuration

  1. Install R on your server, here are steps to install R v4.x (I was able to get 4.3.2) https://medium.com/@hpgomide/how-to-update-your-r-3-x-to-the-r-4-x-in-your-linux-ubuntu-46e2209409c3
  2. Create a static site via Laravel Forge from the screenshot below
    • The domain does not really matter, use any custom sub-domain, even if it does not exist since we shall not be accessing this site anyway
    • Select Static HTML/Next.js/Nuxt.js
    • Select “website Isolation” where you specify a user name for the site, I tend to do this alot allowing me to do custom commands without using the global forge user
    • Create a new static site with Laravel Forge
  3. Install your site code repository, GitHub/Gitlab are natively supported, and uncheck the “Install Composer dependencies”, we do not need this for R
    • Install Repository
  4. The magic happens in the Deployments tab which is enabled once you have connected the code repository
    • Firstly “Enable Quick Deploy” so that whenever you push code to the selected branch in Step #2 above the code is deployed
    • Customize your R script to be executed, in my case it was a simple as in the image below. I would also recommend keeping it as simple as you can to reduce deployment headaches
    • R execution script
  5. You can also test a deployment by clicking the “Deploy Now” button at the top of the page which will re-run the last deployment

Laravel Forge is $12 per month for a single VPS, then costs $19 per month for unlimited servers

17 Feb

TechTip: Secure Access to S3 Folder

Posted by ssmusoke in aws, devops, tips and tricks. Tagged: .

This is more a reminder to myself, so that I do not always have to struggle. Amazon S3 is probably the cheapest cloud storage service, that exists, and in these days of multi-tenant architecutures, provides a great way of accessing data from multiple instances.

However in some cases, you need to store files in S3 that are not viewable to anyone in the world, only to your app. I am hoping that this guide will help with that, and I will keep improving it with feedback and additional learning

The steps are as follows:

  1. Optional: Create an IAM group for users to help in user management
  2. Create an IAM user for each app environment with programmatic access to provide isolation from all other users who share the AWS account, I recommend creating separate users for dev, qa and production environment
  3. Generate access keys for each user which will be used to configure the app
  4. Create an access policy for each environment to restrict access to a bucket or a collection of buckets like below for restricting access to only the dev bucket
    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::project-dev"
      ]
    },
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::project-dev/*"
      ]
    }
  ]
}
  5. Create S3 buckets with no public access, but with the names matching what is defined in the policies
  6. Test adding files to S3 using the credentials to confirm access (I tend to use the AWS cli with profiles for this case)
    

    aws s3 ls s3://project-dev –profile project-dev
    

    7. 



UPDATE 1 – February 24, 2019: Added a poilicy for being able to read the bucket which is different from the bucket contents, see Sid: VisualEditor1.



		
	

			

		


	
		

			

				16
				Oct
			

			
Application and Data Security in the Software Development Lifecycle

			

			Posted by ssmusoke in APIs, devops, opensource, software delivery, software development lifecycle, tips and tricks.			Tagged: , , .			

			

				
This is presentation to Computer Engineering 4th year undergraduate students at Makerere University, Faculty of Technology, College of Design, Art and Technology to introduce them to concepts and approaches for application and data security in the software development process.




Looking for ideas on what I may have missed or omitted to help make this presentation more useful for others

			

		


	
		

			

				27
				Jan
			

			
Software Delivery Skills Plan 2018

			

			Posted by ssmusoke in APIs, devops, opensource, software delivery, software development lifecycle, test driven development (TDD).			Tagged: , , , .			

			

				
If you fail to plan, you are planning to fail! ~Benjamin Franklin


A new year is upon me, and looking over the horizon I am looking to do the following work streams to help better my development skills


1. Work with a new age Javascript framework – vue.js seems the rage, this is also working with webpack and new Javascript build tools


2. Make docker part of my development workflow – this will be project based


3. Distributed ledger proof of concept – the distributed ledger is the rage now, but what can be achieved to prove its capability


4. API First project – this is a separatation of the backend REST APIs from the front end, may be combined with the vue.js to deliver a working project. I will also look to leverage the OpenAPI


5. Write a paper for a scientific journal leveraging the health informatics work I have been doing over the last 3 years and present it at a conference.

			

		


	
		

			

				20
				Feb
			

			
Alternate Approach to Legal Independent Election Tallying

			

			Posted by ssmusoke in alternative approach, database, devops, innovation, internet, mobile computing, mobile technology usage, opensource, responsive design, software development lifecycle, styx technology group, technology, tips and tricks, web development.						

			

				
The Uganda elections are more or less over with less than 6 hours for the Uganda Electoral Commission (EC) to announce the results for the presidential elections.


Given all the time on our hands, with no social media, the team at Styx Technology Group designed the following alternative approach to independent electoral vote tallying for future elections that provides inbuilt mechanisms for audit and verification of results.


The primary data sources for the process are:


    

  1. Official EC list of polling stations and voters per polling station
    2. 

  2. Photos of the signed election tally sheets from each polling station. To ensure that the photos are not tampered with and provide an audit trail:

      

    • Each photograph has to be taken with information on the camera, the GPS coordinates of where the photo was taken, date and time when the photo was taken which is available in many cameras that share it using the Exchangeable Image File Format (EXIF)
      • 

    • Two separate photos of the tally sheets have to be taken by different cameras
      • 

    • The cameras taking equipment may be registered beforehand to provide validation of the source of the information
      • 

    • The signatures of the returning officers and stamp must be clear and visible in the photo
      • 

    

    3. 



The architecture for the technology solution is as follows:


    

  1. Web based solution accessible via any browser. Due to poor Internet connectivity in many areas of the country, an Android app would be provided to assist in data collection, then data sent once the user gets into an area with Internet.
    2. 

  2. The field officers who capture the photos would also be provided with an option of entering the candidate vote tallies.
    3. 

  3. In the tallying center, candidate vote tallies are entered from the photos received and vote tallies entered by data clerks. In order to reduce errors the following approach would be used:

      

    • The clerks are randomly assigned photos as they come in
      • 

    • The tally for a station must be entered correctly by two separate data entry clerks, then approved by a supervisor. This process is formally called the two-pass verification method or double data entry.
      • 

    

    4. 

  4. All correctly entered data is shared with the rest of the world for download and analysis.
    5. 



This system is mission-critical having to be available for the entire vote counting period of 48 hours,  so the architecture includes the following paths for data collection:


    

  1.  Multiple access IP addresses and domains for the website in case some are blocked off
    2. 

  2. Any data collected via the Android app can be sent via email to a dedicated tallying center address. To ensure that only data from the app is received and not changed in transit, encryption is used.
    3. 



The inspiration came from a quote by Ghandi “Be the change you wish to see in the world”, disproving the myth that there is no local capability to design and implement such solutions and most of all that such solutions have to be complex.


Looking forward to hearing your thoughts and suggestions…

			

		


	
		

			
« Older Entries