Author Archives: Muhammad

AWS Athena – Default Profile .aws – okta

AWS Athena – Default Profile .aws – okta

The query editor needs a connection and sometimes we want to use the credentials generated in .aws folder in the machine. They might have been generated by any process including Okta.

You need to override the following parameters with simba driver:

Credentials Provider
jdbc:awsathena://;AwsCredentialsProviderClass=com.simba.athena.amazonaws.auth.profile.ProfileCredentialsProvider;

Credentials Provider Argument
AwsCredentialsProviderArguments=default

You would be needing these additional arguments:

  1. Endpoint URL: athena..amazonaws.com
  2. OutputLocation
  3. Database
  4. S3Output Location

Using Azure Compute Instance in VSCode

Using Azure Compute Instance in VSCode

In order to develop machine learning flows through Azure, you first need to login to your Azure account. After installation of the required pre-requisite extensions, you should see option to login to your azure account.

Azure Account Login

Azure Account Login

It gives you an option to connect to your Azure account.

If you are successfully logged in, you should see the subscriptions:

Your compute instance should be available in your workspace:

Python – Anaconda Installation

Python – Anaconda Installation

Anaconda can be downloaded from the following link:

Anaconda Download

Anaconda Download

You can select the version of the installer suitable for your system. Here we are choosing the Mac version.

Mac Installer

Mac Installer

After downloading we can run it through the Terminal.

Anaconda Command Line

Anaconda Command Line

To get a list of available sub-commands run anaconda-h.

Anaconda Sub Commands

Anaconda Sub Commands

Separate log file for connector

Separate log file for connector

The logs for Kafka connectors are generally added to connectDistributed.out. If you want to create a separate log file for your connector then you can update the following log4j.properties file.

../etc/kafka/connect-log4j.properties

Add the following lines to the file:

log4j.appender.connectorNameSpace=org.apache.log4j.RollingFileAppender
log4j.appender.connectorNameSpace.DatePattern=’.’yyyy-MM-dd-HH
log4j.appender.connectorNameSpace.MaxFileSize=10MB
log4j.appender.connectorNameSpace.MaxBackupIndex=10
log4j.appender.connectorNameSpace.File=${kafka.logs.dir}/connect-my.log
log4j.appender.connectorNameSpace.layout=org.apache.log4j.PatternLayout
log4j.appender.connectorNameSpace.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L – %m%n

log4j.logger.kafka.connect.connectorNameSpace=INFO, connectorNameSpace
log4j.additivity.connector.name.space=false

Angular – Progressive Web Apps using Service Workers

Angular – Progressive Web Apps using Service Workers

Service workers enable creating progressive web applications. It is a script running in the web browser supporting caching of the application resources. They preserve the resources even after the user closes the tab and serves them when the application is requested again.

Setting up Angular Project

Let’s first create a sample angular app.

We can use ng serve to verify that the application has been created successfully.

We can open the application in the web browser and it does load successfully.

It must be remembered that ng serve does not work with service workers so we need to host the application externally through a web server. Let’s use http-server for this. It can simply be installed through npm.

Just make sure that you are able to run it successfully:

Now let’s run our sample PWA app using http-server. Here -c-1 would disable caching.

Adding Service Worker Support

Let’s first verify that our application couldn’t be served offline. We can set the browser to appear as offline through Chrome Dev Tools.

Now we can add support for service worker by adding @angular/pwa.

As we build the application using ng build –prod, we can see that the browser is able to load the application offline using Service Worker.

We can also see Network tab to see service worker in action:

go – segmentation fault: 11

go – segmentation fault: 11

Running go after a few days, I notice that it is crashing with segmentation 11 error. In this post, we will try to get rid of this error in order to run go on our machine. Actually, I need this as a dependency to run some other development tool.

Reading through the internet, it seems that the program is attempting to access a memory that it shouldn’t. Let’s see if running with sudo would be of any help. It doesn’t…

As microfocus explains, it can also be caused by mismatched binaries. This is possible that some new development tools have messed my dev environment up. Let’s first remove go and then reinstall it.

Now we can download the latest go version from here. Just to note that we are running this on macOS Catalina:

Let’s download and run through the installation:

Now you can notice that we are successfully able to run go.

Hyperledger Fabric – Terminology

Hyperledger Fabric – Terminology

Hyperledger Fabric is private and permissioned. In order to gain access, the peers need to be enrolled through a Membership Service Provider (MSP) . It has a deterministic consensus algorithm.

Asset

An asset holds a state and has ownership. They are key / value pairs representing a value, which enables to exchange anything with monetary value. They can be tangible or intangible. They can be modified using chaincode transactions on a channel ledger i.e. the state changes of assets are recorded as transaction on the ledger.

Chaincode

Chaincode is a software that defines the asset. It also defines the transaction instruction to modify the asset. The businesses sign off on the business logic. Chaincode allow these business to interact with the ledger.

Membership Service Provider

They are sometimes also referred as Membership Identity Services. They provides IDs to the authenticated participants which can be used to define permission to the ledger.

Access Control List (ACL)

ACL is used for defining authorization on the ledger and chaincodes. They use IDs provided by MSP.

Node Types

There are two node types. They are Peer nodes and ordering nodes. Peer nodes can batch execute and verify transaction. These transactions are ordered and propagated through ordering nodes. These node types provide efficiency and scalability. A consensus protocol is defined for ordering transactions.

Ordering nodes also maintain the list of organizations that are allowed to create channels. This list of organizations is called consortium. This list is kept in the “order system channel”. This list is administered by an orderer node.Orderer also enforce access control on channels i.e. the updates to the configuration of these channels. After update the configuration block is propogated to peers.

Fabric’s Ledger

Fabric’s ledger is based on both blockchain log and current state of database. The current state can be easily queried. The log is maintained to keep asset provenance in order to provide asset creation and state changes by various members.

There is one ledger per channel and each peer maintains a copy of ledger of the channel they are a member of.

Channels

Hyperledger uses channel to provide privacy to the ledger. Only the participants of the channel have visibility to the assets and transactions on the channel. This is unlike other blockchains where all participants have access to the public ledger. So channels are restrictive message paths defining the subset of participants.

Ordering Service Implementations

In the latest version of Hyperledger fabric, there are three implementations of ordering services including Solo, Kafka and Raft.

Multitenant Fabric

A fabric can have multiple consortiums(list of organizations allowed to create channels) rendering the blockchain as multi-tenant.

Hyperledger Fabric Composer & web sandbox

Hyperledger Fabric Composer & web sandbox

Hyperledger Composer is an framework that allows development of Hyperledger fabric based blockchain applications. It must be noted that it has been deprecated with Fabric 1.4 released in August 2019. This is the first long term support release for Hyperledger fabric. It is pledged by maintainers to provide bug fixes for a period of one year after the date of release.

Hyperledger Composer Web Sandbox

Hyperledger Composer also has an online sandbox that allows us to understand the concepts of Participant, Asset, Transactions and Events.

https://composer-playground.mybluemix.net/

Hyperledger Composer web sandbox

Hyperledger Composer web sandbox

Hyperledger Concepts

Hyperledger Fabric Composer has concepts of participants, assets, transactions and events.

Here is a participant created by instantiating SampleParticipant.

And here we have an asset created using SampleAsset. The asset is assigned with a sample assetId as required by the model.

We can create a transaction and publish it, which can later be loaded from history.

In this model, transaction emits an event with details about the older value and the new value being set for the asset identified by assetId. We can see the details about the event. In reality, this event can be used by applications.

Hyperledger Fabric Development using VSCode

Visual Studio Code has a number of extensions allowing the development of Hyperledger fabric based blockchain applications using Hyperledger composer.

Hyperledger Composer VS Extension

IBM Blockchain Platform

IBM Blockchain Platform