Introduction

In the previous post I showed how I trained a Yolov5 computer vision model to recognize Starfleet combadges, and how Yolo’s own command line tool can be used to recognize them in a video.
That is nice for testing the capabilities of the model, however has no practical use.

We can of course use ultralytics python library to use the model in a Python application, but there are times when we’d like to use it some other environment. Maybe we have a legacy application that we have to maintain, and we’d like to add object detection features to it. Whatever the reason fortunately there’s a solution.

This post will show how to do this in C# .NET with an example Windows Forms application.
The reason for this is because I had to create such an application for a course.

Exporting the model

First and foremost we need to export the model to a format that can be used in another environment.

Fortunately there’s an industry standard format for this: ONNX
which stands for Open Neural Network eXchange, and is supported by many libraries.

Yolo also comes with a built-in tool that can export our model in this format, with a simple command:

python export.py --weights D:\projects\python\yolov5\yolov5\runs\train\exp2\weights\best.pt --include onnx

We can visualize our exported ONNX with a web based tool at https://netron.app

Setting up the project

Our little example project is going to use .NET 6 as the dependencies require this.
Because of this and because of Windows Forms we need (or at least strongly suggested to use) Visual Studio 2022.

In the New Project dialog we need to select Windows Forms App (the one that does NOT say .NET framework), and select .NET 6 as a framework.

Installing dependencies

Next we’re going to install the libraries that we need.

The library we’re going to use the model with is Yolov5Net.
It is built on ML.NET which makes that a dependency too. It has a CPU and a GPU version, but we’re going to stick with the CPU version for this example.
Yolov5Net also depends on ImageSharp, and we’re going to use ImageSharp Draw to manipulate images.

Fortunately we can easily install all this with NuGet commands in the Package Manager Console ( Tools – NuGet Package Manager – Package Manager Console in VS2022 ).

Install-Package Yolov5Net -Version 1.1.0
Install-Package Microsoft.ML.OnnxRuntime -Version 1.14.1
Install-Package SixLabors.ImageSharp.Drawing -Version 2.1.2

Building the application

First we need to create a model definition record for our Yolo based model.
This record contains hyperparameters like the labels we use, the image size used by the model, etc.

For this we need to create a new record (it is named ComBadgeModel in my example app) that inherits from YoloModel base record.
If we used the default parameters for training and only changed the labels then
we can just copy YoloCocoP5Model and then just change these 3 parameters:

• dimensions
• labels
• output

The labels we defined before training the model.
The dimensions is exactly the number of labels + 5.
As for the name of the output we can get it by examining the model with the netron.app web based tool.

We also need a form, with a browse button and a PictureBox.
The browse button will trigger a file selection dialog where we can select the image file we’d like to recognize objects on.
The PictureBox will show this image with bounding boxes around the recognized objects.

In the form’s class we have some setting up to do:

Load a font that we want to use:

Font font = new Font(new FontCollection().Add("C:/Windows/Fonts/consola.ttf"), 12);

Load our computer vision model:

YoloScorer scorer = new YoloScorer("best.onnx");

Once this is done we implement the rest of the application logic in the click event handler of the browse button:

Get the filename from the user through a dialog:

OpenFileDialog ofd = new OpenFileDialog();ofd.Filter = "JPG (.JPG)|.jpg";
DialogResult result = ofd.ShowDialog();
if( result != DialogResult.OK )
{
return;
}

Then we can load the image:

Image img = await Image.LoadAsync(fileName);

Once we have the image, we can detect objects on the loaded image:

List predictions = scorer.Predict(img);

Then we can label the detected objects by drawing bounding boxes and drawing the label and the confidence value:

foreach (YoloPrediction prediction in predictions)
{
 double score = Math.Round(prediction.Score, 2);
 
 float x = prediction.Rectangle.Left - 3;
 float y = prediction.Rectangle.Top - 23;

 img.Mutate(a => a.DrawPolygon(
     new SolidPen(prediction.Label.Color, 1),
     new PointF(prediction.Rectangle.Left, prediction.Rectangle.Top),
     new PointF(prediction.Rectangle.Right, prediction.Rectangle.Top),
     new PointF(prediction.Rectangle.Right, prediction.Rectangle.Bottom),
     new PointF(prediction.Rectangle.Left, prediction.Rectangle.Bottom)
     )
 );

 img.Mutate(
    a => a.DrawText($"{prediction.Label.Name} ({score})",
    font, prediction.Label.Color, new PointF(x, y)));
}

Finally we need to display the labeled image in the PictureBox.
In order to do that we have to convert the ImageSharp Image to System.Drawing.Image.
For that we have to extend the ImageSharp image class with some extensions methods that can be found here and should be added to a static class.

Once this is done we can actually convert and display the image in the PictureBox:

pictureBox1.BackgroundImage = img.ToArray().ToDrawingImage();

The result

The application will look something like this in action:

You can find a full example on Github.

Troubleshooting

If you get a System.InvalidOperationException: Sequence contains no matching element exception then you should check that your model parameters are correct.