Category: React

Behind the Scenes of “…”: Part 3- Making it Pretty

by rawkintrevo, posted in big data for noobs, Engineering, React

At my first job, I was hired to do something (for 40 hours a week) that I was able to write a script to take care of in my first week.   I went to my boss and showed them how efficient I was (like an idiot).  Luckily, my bosses were cool, and the industry was structured as such that my company charged our clients what they were paying me times 3. So my bosses wanted me to stick around, and so gave me free range to do things that would “wow” the client.

I learned in my second through 16th week, in general, no one cares how good your code/data science/blah is, if you can’t make it pretty (I was working in marketing, however I feel that the lesson carries to just about everything beyond analytics and code monkeying).

Becoming one of the beautiful people

I started monkeying around with React and Javascript a year or two ago over the Chicago winter, but it turned to spring before I got too far with it.  This summer, I got back into it. My first project was a little diddy I like to call https://www.carbon-canary.com (which actually started out at http://carbon.exposed). Carbon Canary started off with my hacking on the Core-UI for React framework, but fairly quickly out growing it and implementing many of my own hacks. It does web login with Gmail and Outlook, and will read your email for flight receipts and convert those into your carbon footprint. In these days of flight shaming, etc. a handy tool to have.  It will also chart your foot print over time, and let you add a few things “by hand”.

Carbon-Canary.com was a good first step into playing with react.

I also made http://www.2638belden.com which is a little experiment for a property I own and am (currently) trying to rent out and soon will use as a portal for filing maintenance requests.

So why am I telling you about these? Obviously to boost my google search rank.

As a corollary though, it can also be pointed out, that I’m sort of getting the hang of writing React, and recommend all “back end devs” learn enough of this so you can tell the front end people how to do their job.

Becoming ugly again…

So this whole streaming IoT engine thing was for an IBM hackathon. I figured it would be in bad taste to not make the front end with IBM’s favorite design framework, Carbon.

The hardest part about learning Carbon-react was that all of the examples seem to be for some functional version of React, which I have not really seen out in the wild (I’m using object oriented).  Beyond that, it gets real weird with CSS, and other stuff.

I’m looking at my github code now, and I see it’s been a month since I messed with this.

The main things that stand out in my mind are:

  1. Dealing with Carbon was a bad experience.
  2. There are a lot of poorly documented “features” run amok.
  3. If you never knew anything else, this framework might seem passible, but then you would have a hard time learning any other frameworks (as is common in IBM products bc <design choices>).

My solution after a week of fighting with Carbon to make the simplest things work ended up being an accordion menu and some iFrames of the Flink WebUi and Kabana board.

If I were doing this project for real or if I had an infinite amount of time to learn Carbon, I would like to create charts using React (instead of Kibana), which could make it look nice and pro. Having continued on my React journey for another month, I think making a nice looking frontend (sans Carbon) could be fairly easy done, as well as the components for submitting new jobs, etc. (e.g. replacing the FlinkWeb UI i-frame).

Conclusion / Next Time:

I know this was a short one where I mainly just plugged my other sites and dumped on Carbon, but it’s hard to write some deep cuts about a thing you’re just learning yourself.  That said, I don’t really want to be a front end person, I just want to be able to hack some stuff enough to get a series A and pay one.  If you know anyone who wants to give me a series A please have them contact me.

Next time, we’ll return to the land of Trevor having some passing idea of what he’s talking about as we discuss laying out the K8s ingresses to make all of this work on IBMCloud.

Photo Credit: Trevor Grant; https://www.flickr.com/photos/182053726@N05/48632202606/in/dateposted-public/  ;                Subject “Robin Williams Mural” by Jerkface and Owen Dippie- Logan Square, Chicago IL

Behind the Scenes of “Rawkintrevo’s House of Real Time IoT Analytics, An AIoT platform MVP Demo”

by rawkintrevo, posted in big data for noobs, Engineering, flink, How-Tos, IoT, K8s, openwhisk, python, React, video

Woo, that’s a title- amirigh!?

It’s got everything- buzzwords, a corresponding YouTube video, a Twitter handle conjugated as a proper noun.

Introduction

Just go watch the video– I’m not trying to push traffic to YouTube, but it’s a sort of complicated thing and I don’t do a horrible job of explaining it in the video. You know what, I’m just gonna put it in line.

Ok, so Now you’ve see that.  And you’re wondering? How in the heck?!  Well good news- because you’ve stumbled to the behind the scenes portion where I explain how the magic happened.

There’s a lot of magic going on in there, and some you probably already know and some you’ve got no idea. But this is the story of my journey to becoming a full stack programmer.  As it is said in the Tao of Programming:

There once was a Master Programmer who wrote unstructured programs. A novice programmer, seeking to imitate him, also began to write unstructured programs. When the novice asked the Master to evaluate his progress, the Master criticized him for writing unstructured programs, saying, “What is appropriate for the Master is not appropriate for the novice. You must understand Tao before transcending structure.”

I’m not sure if I’m the Master or the Novice- but this program is definitely unstructured AF. So here is a companion guide that maybe you can learn a thing or two / Fork my repo and tell your boss you did all of this yourself.

Table of Contents

Here’s my rough outline of how I’m going to proceed through the various silliness of this project and the code contained in my github repo .

  1. YOU ARE HERE. A sarcastic introduction, including my dataset, WatsonIoT Platform (MQTT). Also we’ll talk about our data source- and how we shimmed it to push into MQTT, but obviously could (should?) do the same thing with Apache Kafka (instead). I’ll also introduce the chart- we might use that as a map as we move along.
  2. In the second post, I’ll talk about my Apache Flink streaming engine- how it picks up a list of REST endpoints and then hits each one of them.  In the comments of this section you will find people telling me why my way was wrong and what I should have done instead.
  3. In this post I’ll talk about my meandering adventures with React.js, and how little I like the Carbon Design System. In my hack-a-thon submission,  I just iFramed up the Flink WebUI and Kibana, but here’s where I would talk about all the cool things I would have made if I had more time / Carbon-React was a usable system.
  4. In the last post I’ll push this all on IBM’s K8s. I work for IBM, and this was a work thing. I don’t have enough experience on any one else’s K8s (aside from microK8s which doesn’t really count) to bad mouth IBM. They do pay me to tell people I work there, so anything to rude in the comments about them will most likely get moderated out. F.u.

Data Source

See README.md and scroll down to Data Source. I’m happy with that description.

As the program is currently, right about here the schema is passed as a string. My plan was to make that an argument so you could submit jobs from the UI.  Suffice to say, if you have some other interesting data source- either update that to be a command line parameter (PRs are accepted) or just change the string to match your data.  I was also going to do something with Schema inference, but my Scala is rusty and I never was great at Java, and tick-tock.

Watson IoT Platform

I work for IBM, specifically Watson IoT, so I can’t say anything bad about WatsonIoT.  It is basically based on MQTT, which is a pub-sub thing IBM wrote in 1999 (which was before Kafka by about 10 years, to be fair).

If you want to see my hack to push data from the Divvy API into Watson IoT Platform, you can see it here. You will probably notice a couple of oddities.  Most notably, that only 3 stations are picked up to transmit data.  This is because the Free account gets shut down after 200MB of data and you have to upgrade to a $2500/mo plan bc IBM doesn’t really understand linear scaling. /shrug. Obviously this could be easily hacked to just use Kafka and update the Flink Source here.

The Architecture Chart

That’s also in the github, so I’m going to say just look at it on README.md.

Coming Up Next:

Well, this was just about the easiest blog post I’ve ever written.  Up next, I may do some real work and get to talking about my Flink program which picks up a list of API endpoints every 30 seconds, does some sliding window analytics, and then sends each record and the most recent analytics to each of the end points that were picked up, and how in its way- this gives us dynamic model calling. Also- I’ll talk about the other cool things that could/should be done there that I just didn’t get to. /shrug.

See you, Space Cowboy.