Dear Moderator,
I've noticed that certain labels in the parts challenge appear to be inconsistent or noisy. Here are a few examples:
1. p-a972892c46400a881537
2. p-161e753dc6fdb8d9e54b
3. p-dd24ccf37e6e74b43863
4. p-a6ca32701c87bdf68c6d
I understand that real-world datasets can sometimes contain noise, and I wanted to confirm whether these noisy labels are an intentional feature of the dataset or if there might be any corrections or clarifications forthcoming.
Created by Oluwatosin Alabi labdeeman7 Dear Organizer,
it seems that the following images are affected by this:
p-a6ca32701c87bdf68c6d.png
p-161e753dc6fdb8d9e54b.png
p-1fd4760712bc6c8154f2.png
p-3186d714f19de8cc1efe.png
p-37c8b1e39ea98e29c25d.png
p-419683514dae6744b7c3.png
p-57d398397e559d10314a.png
p-592af4f0a1ef73e035b2.png
p-6102c6816dc4ce5a1a46.png
p-69fcb25f1d8904643de2.png
p-7231534ae46d3b9ef26c.png
p-7ec2d7bdbe760c447c0b.png
p-8e37361f20bb8bde8b98.png
p-a972892c46400a881537.png
p-b771f5469a73996584fc.png
p-bf350ddc936f0f1f9914.png
p-cec5fd6b31cfc4ff9eef.png
p-d2a932938c735374ea5a.png
p-dc87e11702fc33139461.png
p-dd24ccf37e6e74b43863.png
p-f52ec8d61535cd935719.png
Sincerely,
Georgii @marwankefah - thank you for responding with a solution. We will get back here in the morning hours with our response on what would be appropriate. Sorry for the inconvenience this may cause.
Hi,
I believe the error in the dataset (less than 1% (~0.2%)) stems from the utilization of the "label2rgb" function during the dataset generation process. Specifically, when an image consists of less than three distinct parts (for instance, two parts such as a wrist represented by the color red and a jaw represented by the color green), the usage of "label2rgb" with **order sensitivity** can result in erroneous labels being produced.
pred=[0,1,2,3] # normal
pred=[0,2,3] #erroneous label with label2rgb
To mitigate this (at least during predictions if you are short in time). I use gray2color based on my pre-defined mappings.
```
from gray2color import gray2color
import cv2
# Define the color palette **BASED ON YOUR PRE-DEFINED COLORS TO LABELS**
pallet_cityscape = np.array([[
[0, 0, 0],
[138, 0, 0],
[49, 205, 49],
[255, 214, 0]
]], np.uint8) / 255
# Assuming pred_labels is within the range of 0, 1, 2, or 3
rgb = gray2color(pred_labels, custom_pallet=pallet_cityscape).astype(np.uint8)
rgb = cv2.cvtColor(rgb, cv2.COLOR_BGR2RGB)
cv2.imwrite(output_mask_path, rgb)
```
It is possible that I may be mistaken in my conclusion. If there is any confusion caused by my response, I sincerely apologize.
Could the organizers please clarify whether it is acceptable to generate predictions based on the RGB values presented on the dataset page and below, even for images with fewer than three distinct parts, and if doing so would not cause any issues with the ground truth?
Parts Mask
Class ID Object class RGB value
1 shaft (255,214,0)
2 wrist (138,0,0)
3 jaw (49,205,49)
0 background (0,0,0)
Sincerely,
Marawan Elbatel
Thanks for your prompt reply.
In the labelling,
green represents jaw
red represents wrist
yellow represents shaft.
But in all the examples I highlighted, the jaw is labelled as red (it should be green) and the wrist is labelled yellow(it should be red).
I apologize for using the word noisy, it does not fit what I was attempting to describe.
Sincerely,
Oluwatosin Alabi.
Can you please explain what do you mean by inconsistent or noisy? I checked these images and I didn't quite see the noise or inconsistency in these images. I would like to learn. Thanks!