Sherloq - An Open-Source Digital Image Forensic Toolset

An open source image forensic toolset

Introduction
"Forensic Image Analysis is the application of image science and domain expertise to interpret the content of an image and/or the image itself in legal matters. Major subdisciplines of Forensic Image Analysis with law enforcement applications include: Photogrammetry, Photographic Comparison, Content Analysis, and Image Authentication." (Scientific Working Group on Imaging Technologies)
Sherloq is a personal research project about implementing a fully integrated environment for digital image forensics. It is not meant as an automatic tool that decide if an image is forged or not (that tool probably will never exist...), but as a companion in putting at work various algorithms to discover potential image inconsistencies.
While many commercial solutions have unaffordable prices and are reserved to law enforcement and government agencies only, this toolset aims to be both a powerful and extensible framework providing a starting point for anyone interested in testing or developing state-of-the-art forensic algorithms.
I strongly believe that security-by-obscurity is the wrong way to offer any kind of security service (i.e. "Using this proprietary software I guarantee you that this photo is pristine... and you have to trust me!"). Instead, following the open-source mentality, everyone should be able to personally experiment various techniques, gain more knowledge and share it to the community... even better if they propose code improvements! :)

Features
A Qt-based GUI provides highly responsive widgets for panning, zooming and inspecting images, while all image processing routines are handled by OpenCV for best efficiency. The software is based on a multi-document interface that can use floating or tabbed view for subwindows and tool outputs can be exported in various textual and graphical formats.
These are the currently planned functions [(***) = fully implemented, (**) = partially implemented, (*) = not yet implemented]:

General

• Original Image: display the unaltered reference image for visual inspection (***)
• Image Digest: compute byte and perceptual hashes together with extension ballistics (**)
• Similarity Search: use reverse search services for finding similar images on the web (*)
• Automatic Tagging: exploit deep learning algorithms for automatic picture tagging (*)

File

• Metadata Dump: gather all metadata information and display security warnings (**)
• EXIF Structure: dump the physical EXIF structure and display an interactive view (***)
• Thumbnail Analysis: if present, extract embedded thumbnail and highlight discrepancies (***)
• Geolocation Data: if present, get geographic data and locate them on a world map view (***)

Inspection

• Enhancing Magnifier: apply local visual enhancements for better identifying forgeries (***)
• Image Adjustments: apply standard adjustments (contrast, brightness, hue, saturation, ...) (***)
• Tonal Range Sweep: interactive tonality range compression for easier artifact detection (***)
• Reference Comparison: synchronized double view to compare reference and evidence images (***)

JPEG

• Quality Estimation: extract quantization tables and estimate last saved JPEG quality (***)
• Compression Ghosts: use error residuals to detect multiple compressions at different levels (**)
• Double Compression: exploit First Digit Statistics to discover potential double compression (**)
• Error Level Analysis: identify areas with different compression levels against a fixed quality (***)

Colors

• RGB/HSV 3D Plots: display interactive 2D and 3D plots of RGB and HSV pixel data (*)
• Color Space Conversion: convert image into RGB/HSV/YCbCr/Lab/CMYK color spaces (***)
• Principal Component Analysis: use PCA to project RGB values onto a different vector space (***)
• RGB Pixel Statistics: compute minimum/maximum/average RGB values for every pixel (***)

Luminance

• Luminance Gradient: analyze brightness variations along X/Y axes of the image (***)
• Frequency Separation: extract the finest details of the luminance channel (*)
• Echo Edge Filter: use 2D Laplacian filter to reveal artificial blurred zones (***)
• Wavelet Reconstruction: re-synthesize image varying wavelet coefficient thresholds (*)

Noise

• Noise Extraction: estimate and separate the natural noise component of the image (***)
• Min/Max Deviation: highlight pixels deviating from block-based min/max statistics (***)
• SNR Consistency: evaluate uniformity of signal-to-noise ratio across the image (***)
• Noise Segmentation: cluster uniform noise areas for easier tampering detection (*)

Tampering

• Contrast Enhancement: analyze histogram inconsistencies caused by enhancements (***)
• Clone Detection: use invariant feature descriptors for copy/rotate clone area detection (**)
• Resampling Detection: analyze 2D pixel interpolation for detecting resampling traces (**)
• Splicing Detection: use DCT coefficient statistics for automatic splicing zone detection (*)

Setup
The software is written in C++11 using Qt Framework for platform-independent GUI and OpenCV Library for efficient image processing. Other external depencies are ExifTool for metadata extraction, LIBSVM for forgery detection and AlgLib for histogram manipulation.
Even if the project objective is clear, actually the software is an early prototype, so some functionalities are still missing (see list above) and it can be run only from Qt Creator under Linux. I put it on Github to track my development progress even during the alpha stage, so expect issues, bugs and installation headaches, however, if you want to take a look around, feel free to contact me if you are experiencing problems in making it run.

Screenshots

File Analysis: Metadata, Digest and EXIF

 Color Analysis: Space Conversion, PCA Projection, Histograms and Statistics

Visual Inspection: Magnifier Loupe, Image Adjustments and Evidence Comparison

JPEG Analysis: Quantization Tables, Compression Ghosts and Error Level Analysis

Luminance and Noise: Light Gradient, Echo Edge, Min/Max Deviation and SNR Consistency

Download Sherloq

via KitPloit

This article is the property of Tenochtitlan Offensive Security. Verlo Completo --> https://tenochtitlan-sec.blogspot.com

Related links

1. Tools Used For Hacking
2. Ethical Hacker Tools
3. Hacking Tools Hardware
4. Hacker Tools
5. Nsa Hack Tools Download
6. Hacking Tools
7. Computer Hacker
8. Hacking Apps
9. Best Hacking Tools 2020
10. Hacking Tools Windows
11. Pentest Tools
12. Pentest Tools Bluekeep
13. Hack Tools For Pc
14. Hacking Tools Online
15. Hacker Tools Apk
16. Hack Tool Apk No Root
17. Pentest Tools Free
18. Hacking App
19. Beginner Hacker Tools
20. Pentest Tools Bluekeep

The Experience

I once spent nine days as a guest at a Taiwanese Buddhist nunnery. Their hospitality was incredible and what stuck in my mind was the most amazing food I've ever eaten. They made Chinese meat dishes out of plant protein, most likely because the nuns were brought up on a traditional, Chinese omnivorous diet, and this food met their vegetarian religious restrictions without compromise. It was so good, I questioned it's meatlessness, being a vegetarian at the time.

After a week of this amazing food, I mentioned on the way back from our conference, that I could really go for a pizza, especially because there was a Pizza Hut next to the nunnery in busy downtown Taipei. No matter how good something is, you often long for the tastes of home. You know you'll get that consistent experience, even if it's not great. Consistent beats great sometimes. An older scholar overheard me and slammed me for being so disrespectful as to want pizza when our hosts had been so gracious with their amazing food. When we returned to the nunnery for dinner, awaiting us was glorious Pizza Hut pizza. The heart wants what it wants.

When it comes to hobby game stores, consistency of experience is wickedly hard. You can train your staff to greet customers, provide stellar customer service, create intricate systems to maintain product and service, but in the back room it's another story. In our Game Center, your consistency of experience is kind of in your own hands.

I could pay employees to run games of a particular style and quality, but the games they run would be limited to the customer desire to pay for that experience. Other than convention fees, which they seem to have no problem with, nobody wants to pay $10 cash money for me to run Dungeons & Dragons. $10, times six players, is $60 for a 4-hour session paying someone $15/hour. That's just their labor, not profit or materials, or prep time. This is a traditionally free experience that can cross over into "nominal" fee territory, but a real fee will never really capture the value being provided.  That may change with the mainstreamization of gaming, and someone will certainly point out the "professional" dungeon masters, but it's rare.

So we run the Event Center a bit like a concert hall in which we attempt to host high quality concerts, but with no guarantee the experience will be great. We are concert hall people, not the performers. I've been to great concerts and I've been to concerts where the performers were drunk off their asses, but in neither case did I credit or blame the venue. But in the game trade that's exactly what happens. Sexist comment? Bad DM? Poor hygiene? It will all be a black mark against the store, even though there's not a whole lot we can do about it, other than craft policies, brief organizers, and strictly enforce rules. We are facilitators. We use volunteers. The only other option is the thing doesn't happen.

This chaos is also our strength, our protective armor. The inability to provide a consistent experience, but to only provide a neutral venue is unacceptable to anyone with deep pockets who wants in on this. What happens if something really terrible (actionable by law) occurs? How do we make sure the D&D session doesn't have something inappropriate? How do we actually monetize this space that costs us $6,000 a month? Really, that's what we pay. About $50 a seat per month.

The reality of most D&D sessions is there are a lot of slightly boring ones and then one amazing one, which you tend to remember without remembering the boring ones. D&D especially is a constant playtest, as most people don't run the same adventure twice. Imagine sitting through a bunch of boring movies to get to the great one. That's how it tended to be before the Internet, but people want blockbusters every time nowadays, and they can get them by picking and choosing. All of this inconsistency is why there are no national chains of game stores. Managing the managers and the organizers would be like herding cats. You would have to have a whole department called Program Development to plan and test event structures. Publishers can't even pull this off well with their one game. Plus, as mentioned, the customers would never pay, at least not so far.

Anyway, this is something that keeps me up at night. Labor, as minimum wage here approaches $15 an hour, can no longer be the solution to bespoke experiences. We are fast approaching hard limits that are testing the demands of customers with the reality of what is possible in small business. It may just be the little store, with the passionate owner working for close to zero dollars, will be the one providing the consistently amazing experiences that big stores could only dream of. The rest of us are wondering if we should get a liquor license or hire some circus performers.

HOTT 52 - Weeks 7 Thru 9 - Hordes, Heroes And Scenarios, Oh My!

The latter half of February has been a hectic one for me! GaryCon prep, podcasting, running several games... and still playing HOTT! I just haven't had much time to blog about it.

Later on in this post, I'll get into how I use HOTT (and by extension, DBA) to fight published scenarios from sources such as Neil Thomas's "One Hour Wargames" or Charles Grant's "Scenarios for Wargames."

First, the battles and pictures! 

Week 7 - the Week of the Hordes

I have played with Hordes, but for some reason, I never took advantage of the "replacing hordes" option. I decided to see how it would play out with my Orcs bringing hordes to the fight!

Humans (Militia/Regular)      Spear-General x1 Spear x3 Shooter x2 Riders x4 Blades x2

Orcs (Militia/Regular) - defending Warband-General x1 Warband x2 Shooter x3 Riders x2 Hordes x4 Spear x2

The terrain would be a bit challenging for the attacking humans! That impassable hill split up the forces. The human commander opted to stack the lighter forces to the left, to see how things would play out. The orcs had put their hordes to their left  and the human commander set his blades and spears opposite.

 

To the left flank of the humans, the shooters were quickly cut down by orc archers and by the warbands. The human riders swung around the marsh to attack in the good going but they were pushed back!

On the left flank of the orcs, the hordes marched ahead, with goblin wargs on their flank. Orc spears held reserve to prevent any breakouts. The lines collided and hordes were cut down en-masse by dour blades and spears!



But guess who rolls a 6 for PIPs? The hordes keep coming! Fun!

 

Not wanting to get surrounded, the human riders opted to a strategic retreat and join up with the forces on the other side of the hill. Distance and the hill makes this an expensive retreat, PIP-wise, but forming a column helps! The orcs, seeing the peril, move to reform their own line.

 

The lines form again and smash into each other! With recoils and the warband's penchant for pursuit, the Orc General unit finds himself surrounded! The spears cut down the warband and the Warchief is lost - the Orcs flee the field!

Humans win 6 to 11G.

The hordes made this fun! The orcs rolled a number of 5s and 6s for PIPs, making it easy to bring them back. The orcs started out doing very well with turning back the attack by the riders and shooters, but then the momentum shifted. The humans rolled some great rolls once they reformed the line, and the pursuing Orc General-Warband put himself in a bad position.

Week 8 - the Week of the Hero

This week, I decided to try out a Hero element to see how it played. I put the Hero on the human's side, and I decided to oppose it by putting a Behemoth on the Orcs side.

Humans (Militia/Regular) - defending    Spear-General x1 Spear x3 Shooter x2 Riders x2 Blades x2 Hero x1

Orcs (Regular/Militia) Blade-General x1 Blade x3 Spear x4 Warband x2 Behemoth x1

The humans line up to defend their fort from a force of orcs approaching. The human major details an element of riders to to swing wide as a harassing force. The humans are heartened by the appearance of a local hero and war veteran, Sir Larry, who has come to assist with the defence!



As the lines approach each other, the trolls and Sir Larry eye each other warily.

 

Smash! The lines come together and trade blows! Meanwhile, the riders come sneaking behind, forcing the Orc Warchief to detail a unit to keep an eye on them.

 

The hero and troll trade blows and insults, but neither side lands a telling blow. The orc line begins to crumble, forcing the Warchief to plug the hole. Unfortunately, due to recoils, he finds himself alone and overlapped. A 5v1 roll seals the Warchief's fate as once again, the humans win with a General kill!

Humans win 2-6G.

I was not very impressed with the Hero! He and the troll just pushed each other back and forth the entire combat. Perhaps it would have been more interesting if the Orcs had a Mage to threaten the Hero with bespelling! I'm sure that as I use them more, fun things will happen!

Week 9 - the Week of the Scenario

I've been playing Orcs/Goblins vs. the Humans for 8 weeks now, and the Humans have won each and every time! I thought it was time for a change...

I've written about this book, One Hour Wargames by Neil Thomas. It's one of my favorite wargame books, not only for the simple rules, but for the THIRTY scenarios that Neil lays out in the second half of the book.

Scenarios are one of the ways I love to do solo gaming. Set up the situation and let me at it. Between this book and "Scenarios for Wargames" by Charles Grant, I have many situations to play from!

I rolled randomly on my d30 and came up with Scenario 29 - Shambolic Command. This is a fun scenario! The basic jist is that Blue must defend a hill, but is such an inept commander that they can only order a limited number of troops at any one time. 3/4ths of the force are on the hills, with a reserve on the far end of the board. Red's job, with a smaller force, is to take the hill!

Since the Orcs have been getting beaten up quite a bit, I assigned them to Red as the attackers. Blue would try to hold the hill.

The scenario is written so that Blue is numerically superior, so that meant that the Humans would get a 24 AP army, and the Orcs get only a 16 AP army. To implement the "shambolic command", I decided that the Humans would only get 1 or 2 PIPs per turn (1-3, 4-6). I'll go into more details about how I interpreted the scenario for HOTT after the battle report.

Humans (Regular/Militia)  - defending    Hill defense Knight-General x1 Knight x3 Spear x4 Reserve Riders x2 Blades x2

Orcs (Regular/Militia) Spear-General x1 Spear x2 Shooters x2 Knight x2 Hordes x2

 

The layout is per the scenario. I didn't have one hill, so I laid out a series of hills as a ridge for the Humans to guard. The Orcs arrive and slowly march up the ridge. The Human commander positioned himself on his right to prevent the Orc Hordes from flanking. Those 1 to 2 PIPs made the Humans crawl into position. The Orcs prepared to charge!

 

The initial charge was repulsed, with the Humans holding the high ground. (That +1 for being up-hill made a difference!) The Orc Hordes were defeated, but more swarmed towards the hill! On the Human's left flank, the Orc Knights decimated the Shooters and Knight element. The reserves could only look on, with little leadership (PIPs) available. The Human commander had his hands full just trying to keep the battle line organized!

 

The Orc Knights easily dispatched the Human Riders who flung themselves into a desperate counter-attack. The defending Human line stepped back, preparing themselves for another Orc attack that had been reinvigorated by the arrival of more Hordes!

  

The Orcs begin to roll down the Humans' left flank. Then, seeing an opportunity, the Human Spears swarmed around the Orc's Warchief element and destroyed it! Unfortunately, the Humans had lost too many of their own, the bodies littering the hillside. Thus heartened, the Orcs continued grimly on, although they would feel the effects of the loss of the iron fist of their Chief!

  

The Human commander pulled his few troops into a small line on the last hill, as a remaining Human Rider element was wiped out by the Orc Knights. The remaining reserve Human Blades could only watch helplessly as the Human commander became the only element left on the hill...



... and then he was pushed off as well. The Humans fled the ridge, surrendering it to the victorious Orc force!

This was quite a fun battle! It lasted 32 bounds, but the time passed quickly. I really enjoyed how this felt as it played out. The limitation of what the Human commander could do with only 1 to 2 PIPs was a huge factor. The Hordes also made a big contribution, being able to come back and fill in the gaps, adding their influence by overlaps and preventing (mostly) flanking/closing the door maneuvers. 

The loss of the Orc Warchief hampered the Orcs as well, or this would have ended a lot sooner, I think. It meant that, for the most part, my PIPs were halved as each move would take +1 PIP. The orcs were lucky I was rolling a lot of 5s and 6s.

---

Using HOTT with wargame scenarios from books The One Hour Wargame scenarios are written in a generic enough fashion that they can be used by other rules, but they are obviously tuned for Neil Thomas's rules from earlier in the book. To that end, I had to make several allowances to use this particular scenario with HOTT, as well as adjusting some of the rules of HOTT, here's what I did.

- No HOTT stronghold for the defender.

- No HOTT terrain generation - I used the terrain from the scenario as it calls for.

- This scenario called for 6 Blue units and 4 Red units. Using that ratio, I had Blue/Humans with a 24 AP army and Orc/Reds with a 16 AP army. I have random tables of army lists for both values, so I rolled for which armies to use.

For future scenarios, I haven't considered how to include the more fantasy-based units like Mages, Heroes, Clerics, Gods or Dragons yet, as NT's scenarios are decidedly based on non-magical encounters. I might try it in the future, just to see how it works.

- The scenario victory conditions are usually based on objective and time, as versus HOTT's victory conditions based on number of elements eliminated. This specific scenario called for complete control of the hills by 15 turns.

The OHW rules are based on the assumption of a battle of attrition - once units come into contact, they do not break apart until one side is destroyed! Also, the scenario called that only TWO Blue/Human units could move/fight. I think that the turn limit depends on that aspect of the battle.

I decided to ignore the turn limit as well as the limitation of how many units could fight.  I ruled to have victory happen if either the Blue/Human side was convincingly forced off the hill OR if the Red/Orc side lost 11 AP (approx 2/3ds of 16).

- Reinforcements or Unit arrivals - I stick to the schedule as laid out in the scenario. I allow Hordes to be replaced.

- I implemented the "shambolic command" restriction of Blue only moving/fighting 2 units a turn via PIP limitations - Blue/humans could only get 1 to 2 PIPs a turn. I had all units in contact fight.

Overall, I was pleased with the results! I'm looking forward to doing more scenarios with HOTT.