All detectors are accessible from “Detectors“tab, in the top bar.
You can create a new detector by clicking on the blue “Train a Detector” button, on the top left corner of the Detectors page.
This action will prompt a pop-up window where you can name your detector and select the detection type.
You have two choices when it comes to defining a detector type: Count or Segmentation.
Does the class you are looking after has generally a defined size and/or boundaries?
You should be in “Count” mode if your goal for your detector is to detect and count individual objects. If you imagine yourself as the AI, your goal would be to point at all individual instances of your object with your finger. In machine learning terminology, this mode is the same as “object detection”. However, you will still get the proper mask segment of the object as output (all pixels making the object). If you want a simplified output, such as “oriented bounding boxes” check out the “Output Shape” setting.
You should be in “Segmentation” mode if your goal is more to map areas/regions or in imagery terms, textures. You can imagine the AI’s interaction as brushing or painting over areas of interest that you want it to find.
Detecting individual trees or buildings is Count
Mapping dense forest canopy coverage in terms of area is Segmentation, as mapping the presence of waterbodies in mines.
There are some ambiguous cases where either of the two could be used, but the main thing to remember is that Count does a much better job at separating objects that are potentially close together and that separation is required in order to get an accurate count. So basically it comes down to which metric you are more interested in finding (a count or an area).
In this case with classes having a mix of well defined objects boundaries (e.g. buildings or cars) and open land covers (e.g. tree or road coverage), we suggest to go with “Segmentation” mode to handle the latter.
Note: choosing between count or segmentation does not change the overall model under the hood being still deep learning exploiting convolutional neural networks. It adapts the model architecture mainly towards its last layers and its loss function to better handle the separation of objects. This also involves a speed difference when training or running a detector in Count (longer) or Segmentation (faster) mode.
From the list, you can see some of the info of a detector.
This opens a popup, which contains details about your detectors, like the sharing status, creator and id (useful for interaction via the Public API), but also some editable fields, such as the name and the tag (beta feature, which allows to filter detectors when listing them via the Public API).
The Detector Settings enable you to finely adjust and optimize your detectors to meet specific project requirements and use cases beyond just adding more annotations. See sub-pages below for details on each detector setting available.
Basic settings:
Your detector’s resolution specifies the spatial resolution that all training images in your detector will be automatically scaled to when both training and detecting. This is to ensure object and texture size consistency (in pixels) in both your data set and project imagery for better results. When the image resolutions are changed they are simply scaled (like in a simple image editing software). It does not add new details or information to your imagery (a blurry object will remaining a blurry object after scaling). Therefore, it is usually recommended that you only train on and run your detector on, resolutions that are very close to its resolution setting.
When you first create your detector, the resolution will be set the resolution of the first image you add to your detector. It can be then changed in the detector settings under the ‘Spatial Resolution” setting later on if necessary.
We recommend that you use training images that have similar spatial resolutions. You can then set your detector spatial resolution to the average resolution of your training images.
It can sometimes be helpful to change the detector resolution further, for example if the objects you are detecting are either:
Example
We provide here an example to give an idea of the target resolution setting effects:
The detector when observing the images looks at them by dividing each image into smaller tiles, and two undesired scenarios can happen.
Too high-resolution images
Ideally in a tile, you should be able to see your entire object, plus some of its surroundings. If it’s not the case you can use the resolution detector settings to downscale your images so that your objects fit better.
By downscaling our imagery (so setting a lower spatial resolution setting) we see the following, which is much more understandable by the model:
Too low-resolution images
You may also have the opposite problem where your objects are too small (for example 2px by 2px objects are quite challenging for the detector to pick up). Especially if these small objects are adjacent to each other the problem becomes even more challenging for the detector to separate them. You may want to consider upscaling if that is the case using the detector resolution settings. Currently upscaling is limited to 2x.
With the native resolution, we get the results on the right:
Now we can scale up the resolution by 2x to get the following results, much better:
In order to see these kinds of images applied to your case, you can click on the Training Report icon (green icon) to be found in the top bar.
In this way, you can judge if your objects are too big or too small for the detector and consequently adjust the image resolution.
Anything else you should know?
There are two possibilities for your Detector Type, Count and Segmentation.
There are currently two output shape options available in Picterra:
Note that in either case you can visually your the centers of your results outside of the detector settings in your results view.
The size filter simply filters the results by a specified area (in square meters).
There are 3 different settings
Note that Auto is only available in Count mode.
The training time (or steps) is a more advanced machine learning parameter.
When a machine learning model trains it is sending in batches of image/annotation information to the detector in an iterative manner, one batch at a time, where each batch is a small subset of your full dataset. The number of training steps is basically the number of batches being sent to the model.
If your dataset is large (more annotations, more training areas), it means you should use more training steps so that your detector has the chance to be fully exposed to your entire dataset. By default we set the number of steps to 500 but as a rough rule of thumb, if you surpass 200 annotations you should consider adding more steps. You may also want to add more steps if you have more classes, (we will change this setting automatically as you add new classes)
In the future we will be working on a way to better decide an optimal number of training steps, but for the moment you will have to do a bit of experimentation!
Advanced settings:
Whenever you train a detector you are feeding your training areas to a deep learning model one after the other. These models need a fixed image size to be passed into them, which is just the nature of how these models work in training. Let’s call this fixed size image a “tile”. For us the default tile is a 256 x 256 sample of one of your training areas, which is quite standard in the machine learning computer vision field. If your training areas are larger than this, a single tile will only encompass a subpart of that area. This is fine because we will select a tile from each of your training areas multiple times during training and each time we randomly sample the location of the tile in your training area, the idea being that eventually after multiple training steps all training area contents will be seen.
Training tiles as seen in the training report
However, sometimes this tile size is simply too small. Say your object is too large to fit into a 256 x 256 tile and the full object is never seen in one tile by your model. This is like trying to learn what a human face looks like but you’ve only ever seen separately a nose, or an eye, or a mouth, but never all of it together. It can be achieved, but it makes the task harder. The way we’d get around this problem with Picterra previously was to use the detector resolution setting to scale down your images, which can work fine in many cases but in doing so you also lose detail in your image. If that detail is important in producing correct detections (which depends on your use case), then it may not be such a good idea to scale down that imagery. However, if we just increase the tile size then perhaps we can fit the entire object in a single tile without losing detail, and that’s what the tile size setting does. instead of scaling your imagery down by two, you can increase the tile size from 256 x 256 to 512 x 512 for example.
Here is an example dataset of where this setting has helped in improving the results, producing a full 5% increase in performance (from 63% to 68%) on a difficult and complex problem.
Missing some paving on the left as well as a falsely detected rooftop. Improved results on the right (no rooftop detection at all).
Falsely detected road on the left, correct on the right.
Despite not looking for a specific object (this is a segmentation detector), the tile size is still helpful here because the “paving” in here is defined as basically being not the main road for vehicles, which means the class contains multiple different appearances and textures. Of course there are still rules in what materials are used for which and so scaling the imagery down can be detrimental because you lose detail of the texture of the materials.
However, a major factor in how we as humans would determine whether or not something is pavement is not just by its material but also its surrounding context.
We can tell something is a sidewalk because it’s a narrow linear element next to a larger wider linear element, the road. Since the context is important, having more of it is helpful. So how can we increase the context while still maintaining the texture detail? By increasing the tile size. This dataset in particular could also use more data, and perhaps multiple classes, to improve the score further, but that’s a different story.
It sounds great on paper but of course there are a few drawbacks to be aware of. It may not always perform better as expected. Also it may not even be the details that are important for the model to learn what your object is anyways (in the end it’s not us who decides what details in the images to use during training, it’s the model’s responsibility to do that automatically). As mentioned earlier, it really depends on your use case. In addition increasing the tile size means increasing the GPU memory of your model. A bump from 256 x 256 to 512 x 512 means 4x the GPU memory. This means your training will also take 4x longer. In short this setting can be quite helpful but it’s important to experiment and play around with it to see if it can be of benefit to your detector.
This setting involves the feature extractor portion of the network. Deep learning models have a start and an end. The start is where you provide the input to the model (the images) and the end is where you get the output (the detections). At the start of the network there is a sub-network dedicated to extracting features from your imagery. Let’s not confuse this with the word “feature” used in GIS lingo, which refers typically to actual objects or elements in your image. A feature here is basically a list of numbers per pixel that encodes some kind of information about the image, they are defining characteristics of your images and objects of interest, but turned into something less qualitative and more quantitative. It’s using these features that the rest of the deep learning model decides whether or not a pixel belongs to your object of interest or not and from that can produce detections.
The backbone model is exactly this first feature extractor sub network and it can be helpful sometimes to swap it out with different sub networks. Our default backbone is Resnet34. There are variations of this backbone model that we can use as well, like Resnet18 and Resnet50, the latter being a lighter faster to train model meant for smaller datasets, whereas the former being heavier, slightly longer to train, but also better for more complex scenarios and larger datasets. It really shines during multiclass applications where your detector needs a bit more bandwidth to help separate similar looking objects into many different classes.
Here is an applicable case where we have a 10 class dataset on planes, where indeed some of the classes can look quite similar. We trained the detector on both Resnet34 (the default) and Resnet50. Let’s compare some of the mistakes that the Resnet34 version made to the corrected results using the Resnet50 backbone.
Resnet34 on the left, these should all be the same class of plane. On the right we see results from switching to Resnet50, where the model was able to better learn the difference between similar plane types and correctly classify all the planes in this area.
These planes should all be classified into the “cyan” class. On the left with Resnet34 you see that it mistakenly classified most of them into the “yellow” class which was a plane that was similar in size but a different wing angle. On the right, you see that with the Resnet50 model most of them managed to be corrected.
In the future we may consider adding more different varieties of backbone models, such as Efficient Nets but these will require further testing to see whether or not they are actually worth adding in terms of potential performance gains. What we’re trying to avoid here is having a long laundry list of backbones just for the sake of having them.
Background sampling ratio defines how many tiles per batch (always set to 32) are dedicated to sampling over background. What does this mean exactly? In the FAQ about the training tile setting we explained that we randomly sample tiles from within your training areas. Some tiles or regions of tiles will contain fewer annotations and more background than others, or perhaps no annotations at all. It’s important to strike a balance between how many foreground and background samples you have to make sure your detector sees a proper amount of each. This “proper” amount however is something that can vary from use case to use case. It is highly dependent on the complexity of your imagery and what your detector is having difficulty with. For example, if you are having trouble getting a lot of false positives (bad precision) maybe you want your detector to sample more background regions to make it understand better that these regions are really just “background” and shouldn’t be detected. Or perhaps you’re having the opposite problem with false negatives (bad recall), then maybe you want to decrease this value so that you see more “foreground” (or annotated) pixels. As before it’s not an exact science, but something to experiment with if you see a clear trend in how your detector is outputting results or a large difference in the precision and recall scores that’d you’d like to bring closer together.
In this next example the client was attempting to detect small bodies of water in a mountainous region (the alps) from low resolution satellite infrared imagery. The bodies of water themselves are not particularly complex objects. They basically just look like dark blue blobs. But we need to make sure that we don’t falsely detect any other “dark-ish” looking blobs as lakes. Comparing the results of the background sampling ratio at 0.1 to 0.5, we can see a clear difference in results.
Using a 0.1 background sampling ratio
Using a 0.5 background sampling ratio
We can see that the IOU score got a pretty big bump, but in addition the precision also had an increase of about 5%, which means we probably have less false positive detections. Comparing the results visually we do indeed see examples of this.
False positive detections on the left using a background sampling ratio of 0.1, not present on the right where we use 0.5.
False positive detections on the left using a background sampling ratio of 0.1, not present on the right where we use 0.5.
Training a detector is an iterative process, influenced by the nature of your project, resolution of your imagery, clarity of the object of interest, and accuracy level you aim to achieve. Here are the basic steps to follow in order to create an efficient detector:
The training toolbar and training sidebar allow you to create annotations and areas used by the machine learning model, to learn from.
In the Training toolbar on the left, you’ll find the commands to add annotations (Drawing tools and Magic wand) and create areas (Training, Testing and Accuracy areas). It will allow you to import polygons (Import) and add comments (Markers) directly on your dataset.
The sidebar on the right, allows you to manage your layers (Classes), comments (Markers), and provides you insights for improving your model (Warnings and Dataset recommendations).
Multi-class extends the detector building process in Picterra by enabling the annotation of multiple types of objects/segmentations within a single detector. Unlike the traditional method of creating individual detector for each class, multi-class functionality empowers users to include diverse object or segmentation types within a unified workflow.
In addition to workflow improvements it also produces both cleaner (no overlapping detections between classes) and better overall results in terms of accuracy, as the detector can learn to better differentiate between similar looking classes.
For a more in-depth guide with visuals, you can read about it in our multiclass blog post.
Note that the way the model will be fed with training patch of images during the training steps is by sampling uniformly across the classes. Therefore a less represented class (in terms of pixels covered) will be more sampled than the others. The training report shows the “Training area sampling stats” which will allow you to see how the sampling performed and which classes were seen during training (more info about training report). Finally, the advanced detector setting of “Background Sample Ratio” remains valid in controlling how much of pure background tiles vs. foreground (classes) are seen during training. (more info about Advanced Detector Settings)
Picterra provides two pathways for change detection:
This approach works best when image detections are accurate, and if it involves detections of objects disappearing, appearing, or moving.
1. Detect in “before” and “after” images
To compare two images, we detect objects/textures in both the “before” and “after” images and compare them. If an object disappears in the second image, it counts as a disappearance, and if an object appears in the second image, it counts as an appearance. For moving objects, we determine if something has moved based on the overlap between the locations of the two objects. Note that this means the object is shifting / rotating slightly such that there is still overlap between the detection on the “before” and “after” images so that we can still match the object (If it moves too much it could look like one object appearing and another disappearing)
2. Compare detections
In a second step, the detections from the “before” and “after” image are used to identify disappearances, appearances or moving objects. This identification happens by feeding the “before” and “after” image to an Advanced Tool. Note that the Advanced Tool for comparative change detection analysis is not available by default, but can be enabled on your account on request.
3. Change alerts dashboard (optional)
Contact your customer success representative to discuss options for a custom dashboard which allows manual review of automatically detected changes.
1. Combine “before” and “after” images
Use the Format images for change detection Advanced Tool to create a “change” image that can be used to train your detector. This will require two images, which should cover the same geographical location and have the same number of bands.
Contact your customer success representative to enable this Advanced tool if you don’t have it available yet.
2. Set up band settings
Open the added image and hit the Edit bands button. This will open the multispectral dialog in which you should edit the band settings for each image.
A typical change detection setup would have 2 band settings per image:
After doing this for every training image, you are ready for the next steps.
3. Labelling & Training
The annotation process is similar to the usual detector training flow. Note however, that only changes should be annotated.
For instance, if you are interested in buildings that were built between two dates, do not annotate all buildings, but only those buildings that aren’t there when viewing the “before” bands and appear when viewing the “after” bands. The model will learn to identify differences that are similar to the ones you annotate.
Band settings display options
There are multiple ways to configure which bands are displayed:
4. Running the detector
Once the change detector has been trained, it can be run on more images just like any other detector, see Generate Results section for more information.
The Detector Insights tools are crafted to streamline the iterative process of refining and enhancing your models, ensuring highest quality results. You can access a range of advanced Detector Insights tools conveniently located in the menu panel of the Detector Training UI.
Localized training warnings are designed to offer you precise feedback on potential issues or mistakes in your dataset. Rather than sifting through tons of data to locate a small discrepancy, these warnings will directly point you to the problematic areas. Find out more about warnings
Please note that there are also some warnings in the training report but these are not localized on your imagery.
What is dataset recommendation?
Dataset recommendation is a tool that helps you during the training phases of your detector. It generates markers similar to the markers you can create yourself. However, these auto-generated Recommendations markers denote regions where your dataset could use more coverage. Wherever there is a recommendation marker, you will want to consider adding a training/accuracy area centered around this marker.
Another valuable tool for identifying areas where model improvement may be needed is the confidence map.
The confidence map displays a heat map overlay onto the images, with brighter colors indicating higher uncertainty levels in the model’s detections. This could represent detections the model is unsure about or ones it almost detected but lacked confidence to output. By comparing these heat maps to the results, you can deduce where your detector may be encountering confusion.
How to activate a confidence map? To activate a confidence map, simply click on a button in the area panel for an immediate view of the model’s confidence levels in these specific zones.
How to use confidence maps? The main benefit of confidence maps is their ability to pinpoint areas in the dataset or detector model that may require refinement. Start by examining accuracy scores in specific areas. If the scores are low, use the confidence maps to delve deeper and understand the underlying issues.
How to interpret confidence maps? It’s important to acknowledge that some uncertainty is normal, especially around object edges, which are naturally more ambiguous for detection models and you can safely ignore those. Focus on significant bright spots within the maps, indicating areas of higher uncertainty and potential for model improvement.
The training report is available after each training iteration is complete and can be accessed using the button on the right of the “Train Detector” button as seen below.
The training report is an extremely useful tool to give you more detailed insights into your detector and help you with improvements and additional experimentation. It contains multiple sections detailed below.
In more details, here are the different situations you can be facing:
Detector merging simply copies every raster (along with its annotations/areas) belonging to each of the “source” detectors into the “target” detector. At this point, when training the latter, our ML model will learn from all this information, plus the original one (on target).
In case you need to annotate in parallel on a single detector, the suggested workflow is
Picterra platform allows multiple users the ability to work on the specific detector simultaneously. All detectors shared across the organization support the collaboration feature which allows multiple users to train and annotate the detector.
