ASP Web API with Azure Table Storage

Using ASP Web API with Azure table storage

We've already covered the basics of creating an ASP Web API project and the basics for Azure Table Storage

ASP WEB API intro

Azure Table Storage

For the sample code for this check out our GitHub

Now we're going to get into using both of these. Creating a simple ASP Web API to create data into an Azure Table storage and to get that data back out. We'll do some testing of the API with Postman

As earlier we're using Visual Studio 2015.

Lets get going

Create a Web API project

Create Web API Project

Add Azure Storage from nuget.

We'll create a new controller building on our last post on Azure Table storage. We'll go for a "PlayerController"

Create API Controller

Select "Web API 2 Controller" from the Add scaffold option.

In the model create an entity for Table Storage "PlayerEntity". The player entity will look the same as we had in our last post.

Player Entity

Now back to our controller, we want to create two methods a POST method to create a player and a Get method to get players for a sport.

    public List<PlayerEntity> Get(string sport)
    {
        List<PlayerEntity> _records = new List<PlayerEntity>();

        //Create a storage account object.
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);

        // Create the table client.
        CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

        // Retrieve a reference to the table.
        CloudTable table = tableClient.GetTableReference("player");

        // Create the table if it doesn't exist.
        table.CreateIfNotExists();

        // Get All Players for a sport
        TableQuery<PlayerEntity> query = new TableQuery<PlayerEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, sport));

        foreach (PlayerEntity entity in table.ExecuteQuery(query))
        {
            _records.Add(entity);
        }

        return _records;
    }

This is the same as we used in our last post on Azure table storage.

We create a storage account with a connection string.

The table client is created and this we check for the table "Player" if it doesn't exist we create it.

Azure Table storage is a partitioned database. For this demo we create a partition based on Sport. So we can create players for "Soccer", "Rugby" and "Hurling" and so on. The get method will then only return the players for the sport we want.

For the Post there are a few ways to pass parameters we'll keep it simple for now. We'll pass the parameters into the POST method.

    // POST api/Player
    public string Post(string sSport, string sRow, string sFirstName, string sLastName, string sClub, string sPostition)
    {
        string sResponse = "";

        // Create our player

        // Create the entity with a partition key for sport and a row
        // Row should be unique within that partition
        PlayerEntity _record = new PlayerEntity(sSport, sRow);

        _record.Sport_VC = sSport;
        _record.First_Name_VC = sFirstName;
        _record.Last_Name_VC = sLastName;
        _record.Club_VC = sClub;
        _record.Position_VC = sPostition;

        //Create a storage account object.
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);

        // Create the table client.
        CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

        // Retrieve a reference to the table.
        CloudTable table = tableClient.GetTableReference("player");

        // Create the TableOperation object that inserts the customer entity.
        TableOperation insertOperation = TableOperation.Insert(_record);

        try
        {
            // Execute the insert operation.
            table.Execute(insertOperation);

            sResponse = "OK";
        }
        catch (Exception ex)
        {
            sResponse = "Failed: " + ex.ToString();
        }
        return sResponse;
    }

With this Post method, we can take the parameters passed in and then create a player in our Table storage.

Testing with PostMan

We will use Postman to do the testing if you don't have it go get it and check out this post on using Postman

Our Get method is simple it takes a parameter of sport and then will return all the players in that sport.

In post man we create a GET Request to

http://*LOCALHOST*/api/Player?sport=soccer

This will return a list of all players in Soccer.

To test the POST method change the method in Postman to "POST"

http://*LOCALHOST*/api/Player?sSport=soccer&sRow=52&sFirstName=Chris&sLastName=Smalling&sClub=Man Utd&sPostition=CB

That's it, since we kept our POST simple we just need to pass the parameters and we can add the player to the Azure table.

Coming soon we'll start to work on our POST method and passing the parameters in different ways.

For the sample code for this check out our GitHub

Using ASP Web API with Azure table storage We've already covered the basics of creating an ASP Web API project and the basics for Azure Table Storage ASP WEB API intro Azure Table Storage For the sample code for this check out our GitHub Now we're going to get into…

Read More

Azure Table Storage

About Azure Table Storage

Windows Azure Table storage is a service that stores structured data in the cloud. It's like NoSQL but with more of the SQL, for example it does not store the data as a Document but as rows. You still get the flexibility of NoSQL storage.

Table storage is a non-relational, key-value-pair, storage system suitable for storing massive amounts of unstructured data. Whereas relational stores such as SQL Server, with highly normalized designs, are optimized for storing data so that queries are easy to produce, the non-relational stores like Table Storage are optimized for simple retrieval and fast inserts. Table storage is schemaless, it's easy to adapt your data as the needs of your application evolve. Access to data is fast for all kinds of applications. Table storage is typically significantly lower in cost than traditional SQL for similar volumes of data.

You can use Table storage to store flexible datasets, such as user data for web applications, address books, device information, and any other type of metadata that your service requires. You can store any number of entities in a table, and a storage account may contain any number of tables, up to the capacity limit of the storage account.

Pricing

Windows Azure table storage is cheap. All you pay for the number of transactions and the amount of storage you need. So no licences, no servers, no VM's just what you need when you need it.

Azure Table pricing

This is the pricing in EUR for North Europe for more go here and select "Table" as the storage to see the pricing.

So what that means is that if I want to store 100 GB of data and access it 200,000 times. I pay less than EUR 6. That's great however you look at it. It means I can store a massive about of data and have quick ready access to it and not have to worry about the cost.

Getting started

To get started, open your Azure Protal and select "New" filter to "Storage account" and select "General Purpose" provide a name and select the "Resource Group"

Create Azure Storage Group

Once you hit create you have to wait for it to deploy and create the storage account.

Azure Storage Group Deployed

Once it's deployed go ahead and open it up. Selecting "Tables" will show "no tables found" in the tables blade.

We need the Access keys for be able to do anything. In the main storage blade select "Settings" and click to open Access keys this will show you a "Key1" and "Key2"

Azure Storage Access keys

We'll need them next.

Do some code

For the sample code github

In Visual studio create a new Console Application

Install the Azure Storage nuget package

Now in the main cs file lets bring in the Azure Storage

using Microsoft.WindowsAzure.Storage; using Microsoft.WindowsAzure.Storage.Table;

For Azure Storage our connection string looks like this

DefaultEndpointsProtocol=https;AccountName=storagesample;AccountKey=storagesamplekey

Where StorageSample and StorageSampleKey would be replaced with the account name and the key that you want to use.

For this demo I created a static string and loaded my connection string into it.

Next we need to create our entity. Create a new class 'PlayerEntity'

In this class we'll turn it into a Table Entity

using Microsoft.WindowsAzure.Storage.Table;

public class PlayerEntity : TableEntity

The rest of our Player class should come together like this

public class PlayerEntity : TableEntity
{
    public PlayerEntity(string skey, string srow)
    {
        this.PartitionKey = skey;
        this.RowKey = srow;
    }

    public PlayerEntity() { }

    public string Sport_VC { get; set; }
    public string Club_VC { get; set; }
    public string First_Name_VC { get; set; }
    public string Last_Name_VC { get; set; }
    public string Position_VC { get; set; }
}

Back over to our Program.cs, lets add some code to do something.

    static void Main(string[] args)
    {
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);

        // Create the table client.
        CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

        // Retrieve a reference to the table.
        CloudTable table = tableClient.GetTableReference("player");

        // Create the table if it doesn't exist.
        table.CreateIfNotExists();

        // Get All Players for a sport

        TableQuery<PlayerEntity> query = new TableQuery<PlayerEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "soccer"));

int iCount = 0;
        foreach (PlayerEntity entity in table.ExecuteQuery(query))
        {
            Console.WriteLine("Club:" + entity.Club_VC + " Player Name: " + entity.First_Name_VC + " " + entity.Last_Name_VC + " Position: " + entity.Position_VC);

            iCount++;
        }



    }

Lets walk through this code step by step.

        //Create a storage account object.
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);

This will create the storage account with our connection string.

        // Create the table client.
        CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

        // Retrieve a reference to the table.
        CloudTable table = tableClient.GetTableReference("player");

        // Create the table if it doesn't exist.
        table.CreateIfNotExists();

Create a table client and then we create a table called "player" then we check if it exists. If it does not then table.CreateIfNotExists() will create a new table for us.

Next we'll go and get some data from the table.

        // Get All Players for a sport

        TableQuery<PlayerEntity> query = new TableQuery<PlayerEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "soccer"));

You might have worked out the theme for this now. But this code allows us to create a query to get the players for a sport. In this case "soccer", The partition we created on this table is for Sports. Once we have built the query we can execute it.

int iCount = 0;
        foreach (PlayerEntity entity in table.ExecuteQuery(query))
        {
            Console.WriteLine("Club:" + entity.Club_VC + " Player Name: " + entity.First_Name_VC + " " + entity.Last_Name_VC + " Position: " + entity.Position_VC);

            iCount++;
        }

Here we execute the query which will return records for the Sport and then write them to the console. If you run this application now it will return nothing because we haven't created any data.

So lets create some data
static Boolean CreateData(string sSport, string sRow, string sFirstName, string sLastName, string sClub, string sPostition) { Boolean bSuccess = false;

        // Create our player

        // Create the entity with a partition key for sport and a row
        // Row should be unique within that partition
        PlayerEntity _record = new PlayerEntity(sSport, sRow);

        _record.Sport_VC = sSport;
        _record.First_Name_VC = sFirstName;
        _record.Last_Name_VC = sLastName;
        _record.Club_VC = sClub;
        _record.Position_VC = sPostition;

        //Create a storage account object.
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);

        // Create the table client.
        CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

        // Retrieve a reference to the table.
        CloudTable table = tableClient.GetTableReference("player");


        // Create the TableOperation object that inserts the customer entity.
        TableOperation insertOperation = TableOperation.Insert(_record);

        try
        {

            // Execute the insert operation.
            table.Execute(insertOperation);

            bSuccess = true;
        }
        catch (Exception ex)
        {
            bSuccess = false;
        }


        return bSuccess;
    }

What does this code do,

First of all we create a new method for creating the data.

        // Create our player

        // Create the entity with a partition key for sport and a row
        // Row should be unique within that partition
        PlayerEntity _record = new PlayerEntity(sSport, sRow);

        _record.Sport_VC = sSport;
        _record.First_Name_VC = sFirstName;
        _record.Last_Name_VC = sLastName;
        _record.Club_VC = sClub;
        _record.Position_VC = sPostition;

We create our player Entity and setup the values.

PlayerEntity _record = new PlayerEntity(sSport, sRow)

Will create our table entity for us with the Row ID and the Partition key.
Row ID should be unqiue

TableOperation insertOperation = TableOperation.Insert(_record);

This sets it all ready to go, then we do the table.Execute(insertOperation); to create the record on our table.

So what I've done for this demo is add a iCount int that will hold the record count. Then at the end we add a call to our create data and it will create our first player.

CreateData("soccer", iCount.ToString(), "Wayne", "Rooney", "Man Utd", "Striker");

Once you run the app it will create the footballers and that's it, it's very simple to get started.

For the sample code github

About Azure Table Storage Windows Azure Table storage is a service that stores structured data in the cloud. It's like NoSQL but with more of the SQL, for example it does not store the data as a Document but as rows. You still get the flexibility of NoSQL storage. Table…

Read More

KUDU with Microsoft Azure Web Apps

Kudu is the central nervous system of a Microsoft Azure Web App.

To access your Kudu console, navigate to https://appname.scm.azurewebsites.net. (note: open the kudu console under a new tab in the same browser as you are logged into the azure portal)

Once you open the kudu you should see the console.

Azure Kudu

Environment

Will provide a break down of the system information. The Application information.

KuduEnvironment

Debug Console

This lets you open a cmd prompt or a Powershell prompt. (no really)

Run the command

powershell –command get-process

This will show you the running processes for the Azure Web app.

Process Explorer

It's our old favourite Process Explorer but on our Azure Webapp

Process Explorer

Tools

Tools gives us access to more diagnostic tools.

Azure Kudu Tools

  • Diagnostic Dump
  • Log Stream - This is the log output from our Web app
  • WebJobs Dashboard - If we're using Web jobs
  • Web hooks
  • Download Deployment scripts
  • Support - Someone to yell at
Site Extensions

There are a load of Extensions that can be added to your site. In an earlier post we used site Extensions to add Lets Encrypt to Windows Azure

Installed Extensions

Under Gallery there is a large number of Extensions that can help pimp out your Azure Web App.

Site Extensions

Kudu is the central nervous system of a Microsoft Azure Web App. To access your Kudu console, navigate to https://appname.scm.azurewebsites.net. (note: open the kudu console under a new tab in the same browser as you are logged into the azure portal) Once you open the kudu…

Read More

Lets Encrypt Windows Azure

It was time to do it, it was time to go HTTPS crazy or at least try it and see what I can break.

Lets Encrypt

Lets Encrypt is an SSL Certificate Authority, it's free and automated. Certs are valid for a few months at a time and it handles the renewal.

The two main reasons to avoid going through SSL setup on sites were renewing and the
price. Lets Encrypt has done away with both these issues.

Windows Azure

My first two sites that I was looking at changing ExtractorApp and CloudStatus are both hosted on Windows Azure as web apps.

I found a great Azure Web App Site Extension created by Simon J.K. Pedersen or in Github

Custom Domain Names

Make sure you have custom domains set up and configured on your Azure Webapps.

Getting the Extension Set up

This is a quick step by step to set up.

Service Principal

A service principal is a Azure AD entry that can be used for unattended access to Azure resources. Think of it like a Service Account. You need to create a service principal and grant it access to the resource groups. Lets encrypt site extension will then be able to renew your certificate without manual involvement. The Service principal also makes it easy to install the first time around. 1 service principal can be used over multiple web apps also once it's under your subscription.

Create service Principal.

Login into Azure

Browse for Active Directory, this will open the Classic portal(as of this post being published)

I already had a "Default Directory" so in the Default Directory (if you don't have a directory create one)

In the directory select "Applications" and then add
Add application

Select "An Application developed by my organisation"

Create Step 1

Give it a name, this will also become the name of the service principal, and select WEB APPLICATION AND/OR WEB API and click next.

Create service principal

Set the URI they just have to be valid urls they are not validated or tested.

service principal uri

You now have a service principal. I told you it wasn't going to hurt

Next we need to collect some information

Client ID

The service principal is uniquely identified by its client id to get that, select the Configure tab and copy the Client ID.

Select client

Client Secret

Just below the Client Id on the configure tab is the keys, First select a duration, 1 year, 2 years. Then once you hit save it will generate a key COPY THIS as once you leave this page you can't get it again.

Client Keys

Grant permissions to the Service Principal

Next we need to give permission to the service principal back to the new portal and the resource group. We'll grant the service principal "Contribute" access.

Back to the new portal and open the "Resource Groups"

Filter Resource Groups

Select the "Resource Group" you want and then select "Access"

Select Access

Select Add to add a new user.

Add User

Select a Role - Select "Contributor"

In the add User enter the name of the Service Principal to filter.

Select Service Principal

Select this and then we'll add Lets encrypt


Install the Lets Encrypt Site Extension

Head to the Kudu scm for your Azure webapp.

YOURWEBSITE.scm.azurewebsites.net

Add User

Select "Site Extensions"

Select Gallery and search for "Lets encrypt"

Configure Site Extension

Configure the Site Extension

Once the site Extension is installed follow the steps and restart the website as needed.

Open the Azure Lets encyrypt extension. (click the play button)

Site Extension

  • Tenant - is the tenant name of the Azure AD that the service principal was created in mine was: yourdomain.onmicrosoft.com not sure if that's the same for everyone.
  • SubscriptionId - is the subscription id that the resource groups belongs to.
  • ClientId - is the client id of the application/service principal you saved earlier.
  • ClientSecret - is the key you created for the service principal.
  • ResourceGroupName - is the name of the resource group that you granted access to earlier.
  • ServicePlanResourceGroupName - is the name of the resource group.

Tick the "Update Application Settings" and hit Next

And that's it. You should see a list of host names with a list of "Certificates"

And that's it your done. If you open the Website as https://Domain name.

I was able to use the same service principal for multiple apps within the same resource group.

I've used this for CloudStatus and ExtractorApp

Renewal

The Azure extension handles the renewal of the cert with an Azure Web Job. This has failed on my sites but all I had to do was go into the Kudo and then reset the cert took 30 seconds. I'll dig into why the webjob failed and update this post with those steps. If I haven't and you have an suggestions let me know.

It was time to do it, it was time to go HTTPS crazy or at least try it and see what I can break. Lets Encrypt Lets Encrypt is an SSL Certificate Authority, it's free and automated. Certs are valid for a few months at a time and it handles…

Read More

Windows Azure Explained

Windows Azure

A quick overview of Windows Azure and what it offers, I've explained each of the different services and also included some alternatives so you can see what it's like. This post will get updated from time to time.

What is Windows Azure

Microsoft Azure is a cloud computing platform and infrastructure, created by Microsoft, for building, deploying and managing applications and services through a global network of Microsoft-managed datacentres. It provides both PaaS and IaaS services and supports many different programming languages, tools and frameworks, including both Microsoft-specific and third-party software and systems.

Services
Service Description Alternative
Compute
App Services Run stuff Websites, WebApps. Runs apps in PHP, Node, Java, ASP.NET, ASP Core Heroku, Modulus, CloudFoundry, AppHarbour, EngineYard
Cloud Services Not as light as app services, but not as involved as VM's a nice middle ground. Linode, Rackspace VPS
Virtual Machines What it says on the box. Virtual Machines. Rackspace, Amazon EC2, Digital Ocean
Service Fabric Using cloud services to build out microservices. Takes the hassle out of scaling and updating.
Scheduler / Webjobs Run small jobs that must be repeated at set intervals. No need for VM's Heroku Background Jobs, Cron
Azure Container Service Running docker images on Azure. Using DC/OS or Docker Swarm AWS Container Service, Google Container Engine
Developer Services
App Service / Mobile Apps Authenticate, send messages and store network data for mobile platforms. Heroku, Urban Airship, Parse
API Management Control the traffic and limits of what is passed through an API. AWS API Gateway
Notification Hubs Send push notifications to iOS, Android, Windows and Kindle. AWS SNS, UrbanAirship
Visual Studio Team Services Source Control, Continuous Integration, Testing even does some project management Github, Jenkins, CodeShip, BitBucket
Application Insights Monitor performance and exceptions. Get notified of errors. Newrelic
Storage Services
Blob Storage Windows Azure BLOB storage service can be used to store and retrieve Binary Large Objects (BLOBs), or what are more commonly known as files. AWS S3, Rackspace Cloud Files
Table Storage Windows Azure Tables are a non-relational, key-value-pair, storage system suitable for storing massive amounts of unstructured data. Not Relation storage(EX: SQL) DynamoDB
Queue Storage Service Bus, a message queue in Azure. AWS SQS, RabbitMQ, Sidekiq
Search A fully managed search-as-a-service in the cloud. Azure Search offers scalable full-text search for your apps. AWS CloudSearch, ElasticSearch
Document DB Store JSON NoSQL structured data, works with MopngoDB drivers. DynamoDB, MongoDB
SQL Database It's SQL Server but in the cloud. SQL Server, MySql, Postgres
Redis Store frequently used data in convenient structures. Like Memcache Redis2Go, RedisGreen
Automation Automate Azure services with Powershell AWS Cloudformation
Operational Insights Collect, combine and search logs for issues or troubleshooting. Splunk
KeyVault Keep encryption keys in secure storage, accessible only when used. AWS KMS
Security Center Establish policies for Azure services and alert and report when they aren't met.
Media and CDN Services
Encoding Encode videos into formats useful for things like viewing on mobile, web, 4K HomeTheater demos, etc. AWS ElasticTranscoder
Media Player Actually embed the videos without you having to figure out which player and encodings work for each client platform.
Media Indexer Auto generate text from audio and video files. Search this text or use it for auto generated captions.
Content Protection Protect your videos with DRM.
Live and On Demand Streaming Handle mass live playback of video along with making sure only the right people are watching and that they've hopefully already paid you.
Content Delivery Network Speed delivery of your sites, files, videos to the people requesting them. Cloudfront, MaxCDN
Networking Services
Virtual Network Make it seem like new Azure services just happened to show up on your internal company network. Bring your own IP address. AWS VPC
Express Route Push TBs a day up to Azure without overloading your mom's cable modem? Get a dedicated line into Azure. AWS Direct Connect
VPN Gateway Setup a VPN between your datacenter and Azure to put traffic over. OpenVPN
Traffic Manager Improve performance by putting traffic to datacenters closer to requests and smart failover when a DC catches on fire. AWS ELB
Load Balancer Split traffic between multiple servers/services AWS ELB
Application Gateway Load balance web servers. AWS ELB
DNS It's a DNS service DNSimple, Route 53, GoDaddy
Enterprise / Corporate Services
RemoteApp Put an app on Azure and give users a RDP session to it. Citrix
BizTalk Connect both Azure Enteprise apps (like SAS or Peoplesoft) together. That sounds like fun.
Service Bus Dump commands and data into queuing service that lets you connect lots of devices, servers or client together for better and smoother processing. RabbitMQ, AWS SQS
Backup Keep a data center disaster from wiping out all your backups. Sick of swapping tapes, keeping them offsite and accidently recording hot mixes over the incremental accounting backups? Glacier, Backblaze
Site Recovery Keep a replicated version of your critical network apps on standby in Azure. AWS CloudConfig
Active Directory Setup an Active Directory As A Service that can sync to your Enterprise AD or replace it entirely
Big Data Services
Batch USing batch services online. AWS Elastic Map Reduce
Logic Apps Hook together different pieces to build a cloud workflow for your data. IFTTT, Zapier
SQL Data Warehouse A fully-managed and scalable cloud service. It is still in preview, but solid. Not only is it compatible with several other Azure offerings, such as Machine Learning and Data Factory, but also with various existing SQL Server tools. AWS Redshift
Data Lake Analytics Hold Exabytes of Data, for some reason or another.
Data Lake Store Azure Data Lake Store is a hyper scale data repository for enterprises to build cloud-based data lakes securely AWS ElasticMap Reduce
HDInsight Managed Hadoop, Spark, and R clusters.
Machine Learning Azure Machine Learning enables developers and data scientists to build advanced analytic web services. AWS Machine Learning
Stream Analytics Collect thousands of analytics streams simultaneously. AWS Kinesis
IoT Services
IoT Hub Collect information from your IOT devices. AWS IOT
Pricing

Microsoft have created a really great pricing calculator to work out how much things might add up to. Pricing Calculator

Windows Azure A quick overview of Windows Azure and what it offers, I've explained each of the different services and also included some alternatives so you can see what it's like. This post will get updated from time to time. What is Windows Azure Microsoft Azure is a cloud computing…

Read More