Serverless IoT: How to read telemetry data from IotHub ?

There are several ways to read fata from IotHub. One could be to use Azure Stream nalytics, SPARK or any kind of custom code. When implementing a custom solutions (custom code), you will most likely need more time to implement all feature (i.e.:windowing), which already exist in ASA or SPARK.

But you will get most flexibility. I do not want to recommend any of named solutions, because the decision depends on many factors.

But I definitely want to remark, that implementing of custom code is no more that comnplex as you might think.

Azure Functions offer powerful implementation of event processing on top of EventHostReceiver API.

Here is the code in C#, which demonstrate how to read events:

[FunctionName("ReadStringsFunc")]
public static async Task Run1([EventHubTrigger("iothubconnstrname", 
Connection = "iothubconnstr",
ConsumerGroup = "FunctionApp")]string message, TraceWriter logger)
{
    logger.Info(__aSyNcId_<_FRZMdHjn__quot;{message}");          
}

This is obviously extremely simple. In fact you don't need any plumbing code to hook in receiver process.

But, you might ask about performance of this code. Processing of every single message is kind of costly.

The binding used for Azure Functions is implemented as binding around EventHub receiver, which supports batching.

Following code illustrate how to read batch of messages:

[FunctionName("ReadStringsFunc")]
public static async Task Run1([EventHubTrigger("iothubconnstrname", 
Connection = "iothubconnstr",
ConsumerGroup = "FunctionApp")]string[] messages, TraceWriter logger)
{
    logger.Info(__aSyNcId_<_FRZMdHjn__quot;{messages.Length}");          
}

This is great, but unlikely to JavaScript developers, .NET developers typically work with statically typed messages, for many good reasons.

No problem. You can deserialize messages by yourself or let Azure Function Binding to do it for you.

Here is an example, which shows hot to read batch of messages of type TickerMsg:

[FunctionName("TypedEventReaderFunc1")]
public static async Task Run1([EventHubTrigger("t10iothub", 
Connection = "iothubconnstr",
ConsumerGroup = "FunctionApp")]TicketMsg[] iotHubMessages, TraceWriter logger)
{
    logger.Info(__aSyNcId_<_FRZMdHjn__quot;{iotHubMessages}");
    Console.WriteLine(__aSyNcId_<_FRZMdHjn__quot;{iotHubMessages}");
    foreach (var item in iotHubMessages)
    {
         if(item.ticketId == null) continue;
            await writeToBlob("TicketMsg", item.ticketId, item.entryTime, logger);
    }
}

Deploying to Azure

I have deployed this code to Azure from Visual Studio. The reason for this is to complex management of referenced nuget packages.

For examle, this is the project file, which shows all required references inside of AzureFunction project on top of .NET Framework (NOT .NET Core):

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net461</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.WebJobs.ServiceBus" Version="2.1.0" />
    <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="1.0.8" />
    <PackageReference Include="Newtonsoft.Json" Version="11.0.1" />
  </ItemGroup>
  <ItemGroup>
    <Reference Include="Microsoft.CSharp" />
  </ItemGroup>
  <ItemGroup>
    <None Update="host.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="local.settings.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <CopyToPublishDirectory>Never</CopyToPublishDirectory>
    </None>
  </ItemGroup>
</Project>

All packages shown above must match. Otherwise you might gety many compiling errors.

Following code is the content of function.json file, which is automatically created and deployed by Vsual Studio. You cannot change anything in this file.

{
  "generatedBy": "Microsoft.NET.Sdk.Functions.Generator-1.0.8",
  "configurationSource": "attributes",
  "bindings": [
    {
      "type": "eventHubTrigger",
      "path": "t10iothub",
      "consumerGroup": "FunctionApp",
      "connection": "iothubconnstr",
      "name": "iotHubMessages"
    }
  ],
  "disabled": false,
  "scriptFile": "../bin/StreamFuncApp.dll",
  "entryPoint": "StreamFuncApp.Function1.Run1"
}

But note the binding attribute withname 'connection'. As described in this article, this specifies the name of configuration key of iothub (or eventhub) connection string. This value has to be provided as all other connections string values.

After deployment, all required connection strings (i.e.: storage accounts) are commonly deployed.

Unfortunately IotHub related connections string is not known by Visual Studio. You will have to add it manually as shown at the next picture.

If you don’t do this, Azure Function will be in status “All Right”, but nothing will work.