Hi everybody,
we've been working on our model and discovered an intersting issue yesterday, that we figured we would raise here
This is related to how the images were processed, presumeably when some of the clean up happened. What we noticed is that, while in your normal image viewer, the images appeared in the correct orientation, when we loaded them via a image library (e.g.: opencv), they reversed back to their original orientation
The changes to the images are stored in the EXIF data, and not applied to the actual image data matrix. So you need to make sure, that when you read the images, to also read the EXIF data and apply it correctly, to get the image the way it is supposed to be
You can also read details on why it occurs and how to fix it in this blog post I found: https://medium.com/@ageitgey/the-dumb-reason-your-fancy-computer-vision-app-isnt-working-exif-orientation-73166c7d39da
This is for python specifically, but I am sure you can find similar libraries and apis for R and so forth as well
Many of you might already know this, but I was quite dumbfounded at first when I realized what was going on - so I figured I'd share it here
Created by Michael Stadler stadlerm Actually above problem can be smoothed over by replacing
```
image = Image.open(imagePath)
```
with
```
image= Image.open(imagePath).convert('RGB')
```
It doesn't resolve the multiple TIFF tag error which is a limitation of Pillow, but it accomodates above few images which are black and white single channel. However note procedure above is giving me bad results in 3 cases where the Metadata tag 283 has 2 entries where 1 were expected:
```
UAB427-LF.jpg
UAB428-LF.jpg
UAB472-LF.jpg
```
It is likely that files in test/validation sets will have the same issue. Cheers lars - for us the PIL function works pretty well (image = ImageOps.exif_transpose(image)), but I think it's only available in newer versions
For now we just did a preprocessing step, where we read the images and rewrite them, after applying the transformation According to this bit of code 80 images are flipped or rotated in EXIF, so require transformation when loading:
```
from glob import glob
from PIL import ImageOps, Image
from tqdm import tqdm
import numpy as np
def PIL_to_numpy(image):
return np.array(image)[...,:3]
def PIL_diff(a,b):
return np.linalg.norm(PIL_to_numpy(image) - PIL_to_numpy(image_x))
image_fns = sorted(glob('../../data/train/*.jpg'))
problems = []
for imagePath in tqdm(image_fns):
image = Image.open(imagePath)
image_x = ImageOps.exif_transpose(image)
diff = PIL_diff(image, image_x)
if diff != 0.:
problems.append((diff, image, image_x, imagePath))
problems.sort(key=lambda x: x[0], reverse=True)
print(len(problems))
```
So these in particular, sorted in reverse order of distance between original and EXIF-transformed image:
```
UAB403-RH.jpg
UAB398-RH.jpg
UAB394-RH.jpg
UAB396-RH.jpg
UAB347-RH.jpg
UAB374-RH.jpg
UAB357-RH.jpg
UAB317-LH.jpg
UAB379-RH.jpg
UAB301-RH.jpg
UAB348-LH.jpg
UAB363-RH.jpg
UAB348-RH.jpg
UAB335-LH.jpg
UAB335-RH.jpg
UAB342-LH.jpg
UAB392-RH.jpg
UAB372-RH.jpg
UAB383-RH.jpg
UAB379-LH.jpg
UAB360-LH.jpg
UAB345-LH.jpg
UAB352-LH.jpg
UAB347-LH.jpg
UAB277-RH.jpg
UAB236-RH.jpg
UAB333-LH.jpg
UAB333-RH.jpg
UAB331-LH.jpg
UAB352-RH.jpg
UAB360-RH.jpg
UAB331-RH.jpg
UAB267-LH.jpg
UAB337-LH.jpg
UAB280-RH.jpg
UAB427-LF.jpg
UAB329-LH.jpg
UAB453-LH.jpg
UAB453-RF.jpg
UAB267-RH.jpg
UAB278-LH.jpg
UAB427-LH.jpg
UAB252-LH.jpg
UAB427-RH.jpg
UAB453-RH.jpg
UAB337-RH.jpg
UAB417-LH.jpg
UAB302-RH.jpg
UAB236-LH.jpg
UAB461-LH.jpg
UAB277-LH.jpg
UAB417-RH.jpg
UAB345-RH.jpg
UAB461-RH.jpg
UAB243-LH.jpg
UAB252-RH.jpg
UAB319-RH.jpg
UAB307-LH.jpg
UAB280-LH.jpg
UAB477-LH.jpg
UAB451-RH.jpg
UAB451-LH.jpg
UAB388-RH.jpg
UAB329-RH.jpg
UAB383-LH.jpg
UAB307-RH.jpg
UAB310-LH.jpg
UAB372-LH.jpg
UAB394-LH.jpg
UAB396-LH.jpg
UAB403-LH.jpg
UAB374-LH.jpg
UAB357-LH.jpg
UAB319-LH.jpg
UAB481-LH.jpg
UAB481-RH.jpg
UAB392-LH.jpg
UAB475-RH.jpg
UAB436-LH.jpg
UAB436-RH.jpg
``` [This might work:](https://stackoverflow.com/questions/13872331/rotating-an-image-with-orientation-specified-in-exif-using-python-without-pil-in)
```
from PIL import ImageOps
image = ImageOps.exif_transpose(image)
```
or this which is the most upvoted solution, but longer:
```
from PIL import Image, ExifTags
try:
image=Image.open(filepath)
for orientation in ExifTags.TAGS.keys():
if ExifTags.TAGS[orientation]=='Orientation':
break
exif=dict(image._getexif().items())
if exif[orientation] == 3:
image=image.rotate(180, expand=True)
elif exif[orientation] == 6:
image=image.rotate(270, expand=True)
elif exif[orientation] == 8:
image=image.rotate(90, expand=True)
image.save(filepath)
image.close()
except (AttributeError, KeyError, IndexError):
# cases: image don't have getexif
pass
```
Drop files to upload
Careful with image EXIF data when programmatically loading images page is loading…