Saturday, September 19, 2020

Automating with PowerShell

 This isn't going to be a deep dive on the subject but is presented more as an informational overview of system automation using Microsoft PowerShell.

PowerShell has been around for a while now and has even gone cross-platform in recent years, with the release of PowerShell as an Open Source project that can also run on Linux operating systems. This ubiquity provides flexibility of resources, meaning the people and hardware involved in system automation, and brings a lot of power to bear on a wide range of needs within any size of IT infrastructure.

At a simple level, PowerShell can be used for simple tasks like copying files or even programmatically creating files.  Coupled with the Windows Task Scheduler or Linux cron tab, you can even create regular activities like backups and exports.

At a more advanced level, however, PowerShell gives the system administrator or developer access to everything from Windows Instrumentation to Web Services.  It supports common scripting idioms like function declarations and modules for creating reusable code. It is not, however, a compiled language and lacks advanced concepts like classes, inheritance and namespaces.  But this is what makes it simple to use. No compiler is needed, no building of code to create executable artifacts is required.  At minimum, you need to have PowerShell installed and a text editor and that is all it takes to make it work for you.

My preference has been to use Visual Studio Code, a free development tool from Microsoft, with the PowerShell addon installed.  This, combined with an embedded PowerShell terminal, makes scripting quick and easy.  Combining good coding practices like building test routines, making code modular and reusable, and using a source control system like Git, brings a commercial and industrial level of code management and performance to a PowerShell implementation.  

Given a sufficient set of resources, PowerShell can operate as a regularly executed background task handling repetitious administrative and system management activities, freeing up System Administrators to focus on other areas of concern.  A good rule of thumb for selecting PowerShell as a tool is similar to the way we decide to make a function when writing good code: when you find yourself repeating an action or set of operations, you have found an opportunity for abstraction.  When coding, this means it's a good time to write a callable function. When working as a System Administrator, it's a good indicator the task should be automated, and PowerShell is the perfect, purpose made tool for that automation.

Planning Phase for Scholarships

 I've been thinking about this for a while and have decided this year to start putting the plan in gear.  Datatribe Softwerks, Ltd., will, at some point in the future, be offering scholarships for students seeking technical careers in Information Technology related fields.  The structure, requirements, amounts and other details remain to be determined, but this seems like a great way to start implementing my dream of helping younger talent find their way up and into a field they will enjoy and have a life long passion for.

Meanwhile, if you have comments or suggestions or interest, drop us a note or comment here.  

Thanks!

Monday, January 13, 2020

Constantly Evolving - Mobile Development

When I started writing software that could run on a Blackberry, I was counselled to avoid writing directly to the operating system and pursue a lowest-common-denominator web application to ensure resiliency over time.  This advice paid off marvelously and an 18 month project to create tag-parsing framework to expose Lotus Notes applications uniformly on any web browser has saved a client of mine close to $20 million dollars over 9 years.  When they moved away from Blackberry to iPhone, I only had to update one style rule in a master CSS file.

Recently, however, we have been making increasingly sophisticated applications that require hardware access at a low level on the Android platform, and hence we have begun some limited Android development efforts.  Coming from web development, my impressions of Android development using specifically Android Studio and Java have not been all that favorable.

Android is complex, but not beyond mastery.  The complexity is only one of a few problems I find.  The biggest problem is the constant evolution of the environment.  It doesn't change gradually.  It has changed dramatically and rapidly. This makes researching a solution to a given problem a little difficult.  The internet is littered with examples and discussions dating back 10 years, at this writing. Finding correct version information can be a challenge in google-fu.

One of the other things I would say is a fault is the lack of consistency.  There are numerous classes, plugins, and frameworks for making parts of the UI.  They each have different super classes and hence expose different methods.  Again, not an insurmountable problem, but an annoyance. You might for example, as I did today, be able to find an example for using a SwitchPreference on a PreferenceFragmentCompat, but only an example of using that same SwitchPreference to toggle other inputs on an ActivityCompat or something similar. The pieces are made to be interchangeable, but not universal.  A mix of XML and Java (or Kotlin, as the now formally primary and supported language is) must be used to get the most out of the parts provided.

Then there's Unity, a completely different set of tools for making games.  And now there's Flutter, which throws out all of the above, except Android Studio, and takes a completely different approach to developing for Android more akin to React Native.

It's all well and good. We should certainly grow as we go forward, but we seem to be making the same mistakes over and over again.  Application development is still presented as a coding exercise, when it seems we have long passed the point where we know how to make configuration driven application code generators.  We should be creating new systems that vastly reduce coding effort to get the same, or better, performance and capabilities as yesterday. But we, and I mean techies as a breed, keep churning out new ways to relive the same misery over and again.

I would hope that the industry will grow suspicious of these trends and do better going forward.  I plan to do my part where I can, partly by calling attention to this silliness, and partly by working to make things better.  Hopefully I'll have something to share in that regard in the future.

Monday, December 2, 2019

Reducing Turnaround Time

As more enterprises are migrating to off-the-shelf tools, there is increasing pressure to reduce the turnaround time on custom development.  One way to do this is to use boiler plate.  The next level is to use a boilerplate generator. This basically removes a lot of repeat effort and converts that process into a configuration process rather than a coding exercise.

While the flexibility you might desire is going to be lacking, the time saved is well worth the effort to get setup to do so.  Over the past month, I've been working with two separate technologies to accomplish a certain degree of kick-starting the coding effort.

Spring Roo is a project that has been around for a while an is stable at version 2.0.0.RELEASE.  From a script file, you can procedurally generate a Spring Framework application.  Optionally employing JPA in the configuration script will also populate a target data source table structure with any models you define.  For the price of setting up and understanding Spring Roo, you get a standards based API and data structure rather quickly.  All you have to do is provision the empty data source if not specifying an in-memory RDBMS.

Spring Roo will also create the MVC and service layers for THYMELEAF if you want to have it spit out a UI for you.  The resulting UI is far from complete but can be used to validate the API and then you are free to custom develop the remaining UI tweaks or revisions you desire.

For my needs, though, I would rather use Angular, so for this I have been looking at Angular Schematics.  Schematics is a standalone tool that you can use to create a file tree of code that can be merged into a target tree.  This lets you build components and features as schematics and then push them into an existing Angular CLI project. 

Now, Schematics takes a bit more understanding to come to grips with, but once you understand what it is doing, and how you can take advantage of that, the power it unlocks is fairly impressive. 

As icing on this two layer cake, my latest effort has been to write a front-end script that takes a short config model definition file and a few input parameters and then generate the Roo script as well as automate the execution of the Angular CLI and Schematics calls.  With a carefully designed set of schematics, this provides everything you need to make a complex object repository, API and UI with maybe 30 minutes of specification work.  In the past, doing this all long hand, it would probably take me 2 weeks to a few months depending on what was requested. 

The take away is that now I can focus on the customizations that add value for the customer once the spun-up boilerplate API and UI are ready for them to review.   The nice thing about this approach, from a long term maintenance perspective, is that I can always refine or extend the code generating tool set so it's easy to provide forward compatibility with future features and tool sets.  Careful use of source code management means I can also maintain a version history and recreate past work quickly if and when needed.

Tuesday, March 5, 2019

Linux Fun on Android

It's 2ferWon day! I'm catching up on things I've been stashing away to blog about.

I recently got the python bug due to a request at my main client site. This led me to idly wonder if I could run python on my Android phone. Turns out you can do so, and more.

Thanks to an amazingly cool App, UserLAnd, you can in fact run Linux and hence a bunch of cool tools, including easily working with python, on your very own smart phone or Android tablet.

Once you install UserLAnd, you can pick a shell distribution: Arch, Kali, Debian, Ubuntu and a few others. I chose Debian due to my familiarity with it from past years trying out various linux distros. You set it up by creating a user and password and a VNC password.

Once signed into a user session, you can sudo whatever you like. In my case, I started off by installing python3 and python-pip as well as a host of other libraries Jermaine to the request I had from my client.

If you're a geek like me with an Android device and you're looking to kill a little time, I encourage you to check it out!

Ramping Up with Android

Yes, you may have detected a common thread.  It has been long in the works but my shift away from legacy systems as my bread and butter to cutting edge (or more recent) technology has been a gradual 6 year shift for me.  I've gotten to the point where old-school mastery is no longer something to be grasped on a technology level, but on a practicum and theory level, and in that realm I've gotten to where I am comfortable enough to pickup new technologies as I have need.

Recently, that need turned towards Android.


If you're just starting out with Android Development, I can't recommend highly enough a starter series from Tek Eye. (https://tekeye.uk/android/examples/index) It's now 4-5 years old, and most of the projects will need gradle and build tool updates, but it provides a good, digestible starting point if you are new to Android. (As I have worked through these tutorials and started building my own apps, it is worth noting that Android Studio 3 brings many welcome features, like responsive layouts, that you'll also want to come to grips with).

This offers a basic series of tutorials for Android development. While the downloadable sample code is a little behind the current Android Studio version, it is terse and lesson specific and worth reviewing and running on your own. For use with Android Studio 3.3, the following changes will be needed in your build.gradle files.


But before you make these changes, let the IDE find and fix problems. If asked to upgrade, click OK. If any errors with suggested solutions are displayed in the console, try those suggestions first, like 'try again'.


If asked to recreate the wrapper using the latest supported Gradle version, click OK.





On build.gradle (Module: app), remove the line "buildToolsVersion" altogether.


Change the compileSdkVersion to 26 (or latest)


Change minSdkVersion to 14 (recommended by the console when you first sync the file)


Change targetSdkVersion to 26 (or latest)


Change instances of 'compile' to 'implementation' in the dependencies section.


Change the dependency for com.android.support:appcompat-v7+ to the version of the SDK you are using, eg 'com.android.support:appcompat-v7:26.0.1'



In the build.gradle (Project: myproject) file, add google() before jcenter() wherever you see the reference to the jcenter() repository.


Upgrade your dependency for com.android.tools.build:

classpath 'com.android.tools.build:gradle:3.3.0'

From there, I recommend starting at the beginning of the list of lessons. Download the sample code, unzip it and open the project in Android Studio. Then walk through the written lecture and have a play running the code on your device on an emulator (downloaded through Android Studio).


Time spent on a lesson will depend on you. Downloading and setting up Android will eat some bandwidth and take some time. Generally, Android Studio is very user friendly, but if you get some errors in the console that prevent you from building and running an app, just copy and paste the text into your favorite search engine. There is a lot of help available on sites like Stack Overflow.


After you've worked through the lessons, keep the unzipped code handy. You'll have some familiarity with it and should easily remember where certain examples can be found. Going forward, it can serve as a library of snippets to help you out with your own apps.

Wednesday, January 30, 2019

Coming Up to Speed with Angular 6/7

Over the course of the past few months, I've been making a study of current JavaScript UI frameworks. Angular presently holds the lead as the best fit for my primary client's enterprise environment, so I've been working through a few things to make this a manageable, extensible, quickly deployed solution for a wider swath of development needs. This is a brief discussion of the high-points of that journey. While I'm mostly only taking advantage of Angular 6 functionality, I've been using the Angular 7 CLI to make sure I've got the latest dependencies in the applications I am building right from the start.

POC

The first thing I like to do after nailing down the tool chain for any new technology I am learning is to create a Proof-of-concept implementation. In this case, we had a simple 3 page survey tool that was used once a year for an internal campaign. For the new year, they wanted to add a leader board with bar charts and add email notifications.

Translating the current work from .jsp based web pages to a modular design took a lot of effort, but less time than I thought it would. The first big surgical decision was to separate the UI and the back end web application that provided the APIs driving the user experience. This decision played well into the next section of this article when it came time to break things out into libraries.

The form this separation took, within my Maven project, was that of a multi-module project. Maven is a dependency management and build tool that makes combining external dependencies with your own code a pretty straight-forward, declarative affair. By extension, then, making your own modules should follow roughly the same approach. It does, to an extent, but what you inherit is the responsibility for creating the packaging for your own sub-modules -- a task usually transparent to you when you consume externally provided dependencies.

So in this case, I had separate Maven build files for the API, the UI, and a Web module which simply provided the Web Archive (.war) file packaging at the end of the build. With this arrangement, a web developer can maintain the UI and a back-end engineer can maintain the Spring API separately without excessive concerns for one another's activities.

The POC was a success, and it also created the opportunity to begin identifying and isolating our own unique Angular components as reusable libraries.

MAKING COMPONENTS

Abstraction, if you're not familiar with the term, is the practice (art / science) of breaking a system down into representational components that serve a single purpose in a generic enough fashion as to be useful in many different scenarios. Selecting a function to extract from my POC for this treatment was not my biggest challenge. Getting the process nailed down, from an Angular world view, was what required the most work.

Angular provides external library components to you through NPM. NPM is the Node Package Manager and is by default pointed to a global internet library of Node Modules. If you want to publish your own modules to NPM, you have to sign up for an account. For an enterprise environment, this may not be the best approach. In some cases, components you develop are going to be useless outside of the environment you are working in. In others, they may represent portions of intellectual property that can not be shared. So I had the challenge of working out how to do this without the NPM repository.

The first step was to create a library project. There are several guides available online for this. What this process taught me was the need to carefully decouple my angular components. While Angular makes it super easy to add components to a project with the Angular CLI, you want to be careful what you do with them after that step in terms of how they are connected to parent components. While it is possible, for example, to make components out of form elements, turning one of those elements into a standalone library means you won't necessarily have the parent context while developing and testing that library. Creating them in a way that they have a minimum of interaction with the parent component, then, predisposes them to future extraction into standalone libraries.

The next thing was figuring out how to both put the library under source control and provide a path to being able to package and install it into another future application. This involved three steps.

First, only the "lib" folder in the library component is checked in. Checking in the surrounding test harness is not required.

Second, creating an on-board packaging script for the library allows future users to build it into a .tgz locally after they clone the repository.


Third, using an NPM feature to install from a local file path, the .tgz can be installed as though it comes from the global NPM repository.


This seems like a lot of work, but the important thing here is that it keeps the source code for the library open for extension and modification within the enterprise. I want future developers to be able to fork this library to make their own components or branch the code and possibly submit pull requests when they fix bugs. By keeping the library in a raw, source code format, we get the best features of using packaged libraries and reusable code combined.

NEXT STEPS

Going forward, I have a couple of targets I want to hit with my Angular learning. I have a request to expand the libraries that are documented and tested - essentially create a loose standard for us to follow within the enterprise, so expanding my knowledge of existing external components is going to be a major theme in the coming months. Secondly, I have been delving into Android development on the side. Angular, through Native Script, promises to expedite the process of developing tools that can work on multiple platforms, including Android, so I'll be looking at that.

RESOURCES
Here are some of the resources I've found thus far that have been useful to this pursuit:

Making custom libraries

https://angular.io/guide/creating-libraries

https://medium.com/@tomsu/how-to-build-a-library-for-angular-apps-4f9b38b0ed11

More in-depth example for making libraries
https://blog.angularindepth.com/creating-a-library-in-angular-6-87799552e7e5

https://blog.angularindepth.com/creating-a-library-in-angular-6-part-2-6e2bc1e14121