Sunday, December 4, 2016

Search inside your images using Java

This software will let you search any person or thing from your album,wallpaper,or even from internet websites like facebook.
Our Java program automates the Google Image search so that the image search becomes very simple.

Language Used:
Java/C++

Git Repo:
https://github.com/csanuragjain/extra/tree/master/ImageResolver

Full Software:
https://github.com/csanuragjain/extra/blob/master/ImageResolver/Software/ImageResolver.jar?raw=true
https://github.com/csanuragjain/extra/blob/master/ImageResolver/Software/ImageResolver.dll?raw=true

Pre-requisite:
Download url:
ImagePanel.java :    
https://github.com/csanuragjain/recorder/blob/master/DesktopScreenshotMaker/Code/ImagePanel.java  
MyLogger.dll :    
https://github.com/csanuragjain/recorder/blob/master/DesktopScreenshotMaker/Code/dll/MyLogger.dll  

References:
https://cooltrickshome.blogspot.in/2016/11/create-desktop-screenshot-maker.html https://cooltrickshome.blogspot.in/2016/11/creating-your-personal-keylogger-from.html

POM Dependency:
 <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->  
 <dependency>  
   <groupId>org.apache.httpcomponents</groupId>  
   <artifactId>httpclient</artifactId>  
   <version>4.5.2</version>  
 </dependency>  
 <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpmime -->  
 <dependency>  
   <groupId>org.apache.httpcomponents</groupId>  
   <artifactId>httpmime</artifactId>  
   <version>4.5.2</version>  
 </dependency>  

How it works:

1) Suppose you see an actor on an image on your facebook posts
2) Just Press F8
3) Use your mouse and drag that actor image using mouse
4) Software will upload that actor image to Google and show you the result instantly.
5) For exiting the software simply press F9

Note:
1) As mentioned on pressing F8 a separate frame open up with user current screen background for selecting the image region. In some cases the open frame may get minimized so in those case user will need to maximize the window and then select the region.

Major Module:

Upload method:
  public void upload(File file)  
   throws Exception  
  {  
   final JFrame temp = new JFrame("Searching...");  
   Thread t2 = new Thread()  
   {  
    public void run()  
    {  
     temp.setSize(250, 0);  
     temp.setLayout(new FlowLayout());  
     temp.setVisible(true);  
     temp.setDefaultCloseOperation(2);  
    }  
   };  
   t2.start();  
   MultipartEntity entity = new MultipartEntity();  
   entity.addPart("encoded_image", new InputStreamBody(new FileInputStream(file), file.getName()));  
   HttpPost post = new HttpPost("https://www.google.com/searchbyimage/upload");  
   post.setEntity(entity);  
   HttpClient client = new DefaultHttpClient();  
   HttpResponse response = client.execute(post);  
   String site = response.getFirstHeader("location").getValue();  
   Runtime.getRuntime().exec("cmd /c start " + site);  
   temp.dispose();  
   lock = false;  
  }  

How it works:
1) This module receives the file object pointing to the screenshot of the image to be searched
2) We show a frame to user saying Searching
3) Now we make a POST call to https://www.google.com/searchbyimage/upload and pass the file object we received as argument.
4) The response from google contains the page having result in the location header. We fetch the url from location header.
5) We use runtime to open the url found in previous step
6) The http library dependency are mentioned in POM Dependency section of this blog post.

showFrame method:
  public void showFrame()  
   throws Exception  
  {  
   lock = true;  
   Dimension size = Toolkit.getDefaultToolkit().getScreenSize();  
   Robot robot = new Robot();  
   BufferedImage img = robot.createScreenCapture(new Rectangle(size));  
   ImagePanel panel = new ImagePanel(img);  
   add(panel);  
   setLocation(0, 0);  
   setSize(size);  
   setLayout(new FlowLayout());  
   setUndecorated(true);  
   setVisible(true);  
   addMouseListener(this);  
   addMouseMotionListener(this);  
   setDefaultCloseOperation(2);  
  }  

How it works:
1) When a user press F8 this module is used
2) It takes screenshot of the current user screen and paste that on a JFrame
3) JFrame having the screen background is shown to user where now user can use his mouse to drag the portion of image which he would like to search.
4) This module use ImagePanel which you can get from Pre-requisite section of this blog post.

main Method:

  public static void main(String[] args)  
  {  
   try  
   {  
    JFrame f1 = new JFrame("ImageResolver Help (Exit the App using F9)");  
    JLabel l1 = new JLabel("Please read this before continuing to the program");  
    JLabel l2 = new JLabel("Begin Image Search");  
    JLabel l3 = new JLabel("To begin Image Search,Open the image and then press F8 key");  
    JLabel l4 = new JLabel("A window will open.Just drag the area of the image which you want to search");  
    JLabel l5 = new JLabel("After a small delay the search results will be displayed");  
    JLabel l6 = new JLabel("------------------------------------------------------------------");  
    JLabel l7 = new JLabel("Exit the program");  
    JLabel l8 = new JLabel("For exiting the program Press F9.A confirmation message will be displayed and after that application will close.");  
    JLabel l9 = new JLabel("------------------------------------------------------------------");  
    JLabel l10 = new JLabel("Note: Multiple Searches are not allowed. Please close this window before staring to use the software.");  
    JLabel blank = new JLabel("");  
    JLabel startinfo7 = new JLabel("Software has been Developed by....");  
    JLabel startinfo8 = new JLabel("Anurag Jain");  
    JLabel startinfo9 = new JLabel("Software Engineer");  
    JLabel startinfo10 = new JLabel("(cs.anurag.jain@gmail.com)");  
    JLabel startinfo11 = new JLabel("Project Homepage: https://cooltrickshome.blogspot.in");  
    JLabel startinfo12 = new JLabel("For any problems or feedback you may contact me directly at cs.anurag.jain@gmail.com");  
    f1.add(l1);  
    f1.add(l2);  
    f1.add(l3);  
    f1.add(l4);  
    f1.add(l5);  
    f1.add(l6);  
    f1.add(l7);  
    f1.add(l8);  
    f1.add(l9);  
    f1.add(l10);  
    f1.add(blank);  
    f1.add(blank);  
    f1.add(blank);  
    f1.add(startinfo7);  
    f1.add(startinfo8);  
    f1.add(startinfo9);  
    f1.add(startinfo10);  
    f1.add(startinfo11);  
    f1.add(startinfo12);  
    f1.setLayout(new GridLayout(19, 1));  
    f1.setVisible(true);  
    f1.setSize(700, 700);  
    f1.setDefaultCloseOperation(2);  
    Thread t1 = new Thread(new ImageResolver());  
    t1.start();  
   }  
   catch (Exception e)  
   {  
    JOptionPane.showConfirmDialog(  
     null, "Some Problem Occured.Please try again", "Error",   
     -1);  
   }  
  }  

How it works:
1) We make a Jframe and show how software is going to work
2) We start the thread made in this class so the run method gets called

run Method:
  public synchronized void run()  
  {  
   for (;;)  
   {  
    int value = getKeys();  
    if ((value == 119) && (lock))  
    {  
     JOptionPane.showConfirmDialog(  
      null, "You can only run one query at a time.Let the previous search complete then you may continue with the new search", "Error",   
      -1);  
    }  
    else if ((value == 119) && (!lock))  
    {  
     try  
     {  
      Thread t2 = new Thread()  
      {  
       public void run()  
       {  
        try  
        {  
         new ImageResolver().showFrame();  
        }  
        catch (Exception e)  
        {  
         ImageResolver.lock = false;  
         JOptionPane.showConfirmDialog(  
          null, "Some Problem Occured.Please try again", "Error",   
          -1);  
        }  
       }  
      };  
      t2.start();  
     }  
     catch (Exception e)  
     {  
      lock = false;  
      JOptionPane.showConfirmDialog(  
          null, "Some Problem Occured.Please try again", "Error",   
          -1);  
     }  
    }  
    else if (value == 120)  
    {  
     JOptionPane.showConfirmDialog(  
      null, "Exiting...", "Exit",   
      -1);  
     System.exit(0);  
    }  
   }  
  }  

How it works:
1) This thread keep track of the key pressed by user. It makes use of ImageResolver.dll which you can obtain from pre-requisite section of this blog post.
2) 119 is keycode of F8 and 120 is keycode for F9 as already told in earlier post for keylogger (References section)
3) lock variable make sure that user can perform only one search at a time
4) If user press F8 then showFrame method gets called
5) If user press F9 then program exits

Mouse events method:
  public void draggedScreen()  
   throws Exception  
  {  
   int w = this.c1 - this.c3;  
   int h = this.c2 - this.c4;  
   w *= -1;  
   h *= -1;  
   Robot robot = new Robot();  
   BufferedImage img = robot.createScreenCapture(new Rectangle(this.c1, this.c2, w, h));  
   File save_path = new File("screen1.jpg");  
   ImageIO.write(img, "JPG", save_path);  
   dispose();  
   upload(save_path);  
  }  
   public void mouseClicked(MouseEvent arg0) {}  
  public void mouseEntered(MouseEvent arg0) {}  
  public void mouseExited(MouseEvent arg0) {}  
  public void mousePressed(MouseEvent arg0)  
  {  
   repaint();  
   this.c1 = arg0.getX();  
   this.c2 = arg0.getY();  
  }  
  public void mouseReleased(MouseEvent arg0)  
  {  
   repaint();  
   if (this.drag_status == 1)  
   {  
    this.c3 = arg0.getX();  
    this.c4 = arg0.getY();  
    try  
    {  
     draggedScreen();  
    }  
    catch (Exception e)  
    {  
     e.printStackTrace();  
    }  
   }  
  }  
  public void mouseDragged(MouseEvent arg0)  
  {  
   repaint();  
   this.drag_status = 1;  
   this.c3 = arg0.getX();  
   this.c4 = arg0.getY();  
  }  
  public void mouseMoved(MouseEvent arg0) {}  
  public void paint(Graphics g)  
  {  
   super.paint(g);  
   int w = this.c1 - this.c3;  
   int h = this.c2 - this.c4;  
   w *= -1;  
   h *= -1;  
   if (w < 0) {  
    w *= -1;  
   }  
   g.drawRect(this.c1, this.c2, w, h);  
  }  


How it works:
1) When mouse is pressed we record the starting coordinate in c1 & c2
2) When mouse is dragged then we record the ending coordinate in c3 & c4. We call the paint method which starts drawing the rectangle to show the selected region described by (c1,c2) & (c3,c4)
3) When mouse is released we get the final region for which screenshot need to taken and update c3 and c4.

4) We call draggedScreen method which simply takes the screenshot of the region selected by user and saves it using ImageIO write method.
5) Finally upload method gets called which retreives the result for this image.

Output:





Full Program:

ImageResolver.java
 package com.cooltrickshome;  
 import java.awt.Dimension;  
 import java.awt.FlowLayout;  
 import java.awt.Graphics;  
 import java.awt.GridLayout;  
 import java.awt.Rectangle;  
 import java.awt.Robot;  
 import java.awt.Toolkit;  
 import java.awt.event.MouseEvent;  
 import java.awt.event.MouseListener;  
 import java.awt.event.MouseMotionListener;  
 import java.awt.image.BufferedImage;  
 import java.io.File;  
 import java.io.FileInputStream;  
 import javax.imageio.ImageIO;  
 import javax.swing.JFrame;  
 import javax.swing.JLabel;  
 import javax.swing.JOptionPane;  
 import org.apache.http.Header;  
 import org.apache.http.HttpResponse;  
 import org.apache.http.client.HttpClient;  
 import org.apache.http.client.methods.HttpPost;  
 import org.apache.http.entity.mime.MultipartEntity;  
 import org.apache.http.entity.mime.content.InputStreamBody;  
 import org.apache.http.impl.client.DefaultHttpClient;  
 public class ImageResolver  
  extends JFrame  
  implements MouseListener, MouseMotionListener, Runnable  
 {  
  int drag_status = 0;  
  int c1;  
  int c2;  
  int c3;  
  int c4;  
  static boolean lock = false;  
  static  
  {  
   System.loadLibrary("ImageResolver");  
  }  
  private native int getKeys();  
  public void showFrame()  
   throws Exception  
  {  
   lock = true;  
   Dimension size = Toolkit.getDefaultToolkit().getScreenSize();  
   Robot robot = new Robot();  
   BufferedImage img = robot.createScreenCapture(new Rectangle(size));  
   ImagePanel panel = new ImagePanel(img);  
   add(panel);  
   setLocation(0, 0);  
   setSize(size);  
   setLayout(new FlowLayout());  
   setUndecorated(true);  
   setVisible(true);  
   addMouseListener(this);  
   addMouseMotionListener(this);  
   setDefaultCloseOperation(2);  
  }  
  public void upload(File file)  
   throws Exception  
  {  
   final JFrame temp = new JFrame("Searching...");  
   Thread t2 = new Thread()  
   {  
    public void run()  
    {  
     temp.setSize(250, 0);  
     temp.setLayout(new FlowLayout());  
     temp.setVisible(true);  
     temp.setDefaultCloseOperation(2);  
    }  
   };  
   t2.start();  
   MultipartEntity entity = new MultipartEntity();  
   entity.addPart("encoded_image", new InputStreamBody(new FileInputStream(file), file.getName()));  
   HttpPost post = new HttpPost("https://www.google.com/searchbyimage/upload");  
   post.setEntity(entity);  
   HttpClient client = new DefaultHttpClient();  
   HttpResponse response = client.execute(post);  
   String site = response.getFirstHeader("location").getValue();  
   Runtime.getRuntime().exec("cmd /c start " + site);  
   temp.dispose();  
   lock = false;  
  }  
  public void draggedScreen()  
   throws Exception  
  {  
   int w = this.c1 - this.c3;  
   int h = this.c2 - this.c4;  
   w *= -1;  
   h *= -1;  
   Robot robot = new Robot();  
   BufferedImage img = robot.createScreenCapture(new Rectangle(this.c1, this.c2, w, h));  
   File save_path = new File("screen1.jpg");  
   ImageIO.write(img, "JPG", save_path);  
   dispose();  
   upload(save_path);  
  }  
  public synchronized void run()  
  {  
   for (;;)  
   {  
    int value = getKeys();  
    if ((value == 119) && (lock))  
    {  
     JOptionPane.showConfirmDialog(  
      null, "You can only run one query at a time.Let the previous search complete then you may continue with the new search", "Error",   
      -1);  
    }  
    else if ((value == 119) && (!lock))  
    {  
     try  
     {  
      Thread t2 = new Thread()  
      {  
       public void run()  
       {  
        try  
        {  
         new ImageResolver().showFrame();  
        }  
        catch (Exception e)  
        {  
         ImageResolver.lock = false;  
         JOptionPane.showConfirmDialog(  
          null, "Some Problem Occured.Please try again", "Error",   
          -1);  
        }  
       }  
      };  
      t2.start();  
     }  
     catch (Exception e)  
     {  
      lock = false;  
      JOptionPane.showConfirmDialog(  
          null, "Some Problem Occured.Please try again", "Error",   
          -1);  
     }  
    }  
    else if (value == 120)  
    {  
     JOptionPane.showConfirmDialog(  
      null, "Exiting...", "Exit",   
      -1);  
     System.exit(0);  
    }  
   }  
  }  
  public static void main(String[] args)  
  {  
   try  
   {  
    JFrame f1 = new JFrame("ImageResolver Help (Exit the App using F9)");  
    JLabel l1 = new JLabel("Please read this before continuing to the program");  
    JLabel l2 = new JLabel("Begin Image Search");  
    JLabel l3 = new JLabel("To begin Image Search,Open the image and then press F8 key");  
    JLabel l4 = new JLabel("A window will open.Just drag the area of the image which you want to search");  
    JLabel l5 = new JLabel("After a small delay the search results will be displayed");  
    JLabel l6 = new JLabel("------------------------------------------------------------------");  
    JLabel l7 = new JLabel("Exit the program");  
    JLabel l8 = new JLabel("For exiting the program Press F9.A confirmation message will be displayed and after that application will close.");  
    JLabel l9 = new JLabel("------------------------------------------------------------------");  
    JLabel l10 = new JLabel("Note: Multiple Searches are not allowed. Please close this window before staring to use the software.");  
    JLabel blank = new JLabel("");  
    JLabel startinfo7 = new JLabel("Software has been Developed by....");  
    JLabel startinfo8 = new JLabel("Anurag Jain");  
    JLabel startinfo9 = new JLabel("Software Engineer");  
    JLabel startinfo10 = new JLabel("(cs.anurag.jain@gmail.com)");  
    JLabel startinfo11 = new JLabel("Project Homepage: https://cooltrickshome.blogspot.in");  
    JLabel startinfo12 = new JLabel("For any problems or feedback you may contact me directly at cs.anurag.jain@gmail.com");  
    f1.add(l1);  
    f1.add(l2);  
    f1.add(l3);  
    f1.add(l4);  
    f1.add(l5);  
    f1.add(l6);  
    f1.add(l7);  
    f1.add(l8);  
    f1.add(l9);  
    f1.add(l10);  
    f1.add(blank);  
    f1.add(blank);  
    f1.add(blank);  
    f1.add(startinfo7);  
    f1.add(startinfo8);  
    f1.add(startinfo9);  
    f1.add(startinfo10);  
    f1.add(startinfo11);  
    f1.add(startinfo12);  
    f1.setLayout(new GridLayout(19, 1));  
    f1.setVisible(true);  
    f1.setSize(700, 700);  
    f1.setDefaultCloseOperation(2);  
    Thread t1 = new Thread(new ImageResolver());  
    t1.start();  
   }  
   catch (Exception e)  
   {  
    JOptionPane.showConfirmDialog(  
     null, "Some Problem Occured.Please try again", "Error",   
     -1);  
   }  
  }  
  public void mouseClicked(MouseEvent arg0) {}  
  public void mouseEntered(MouseEvent arg0) {}  
  public void mouseExited(MouseEvent arg0) {}  
  public void mousePressed(MouseEvent arg0)  
  {  
   repaint();  
   this.c1 = arg0.getX();  
   this.c2 = arg0.getY();  
  }  
  public void mouseReleased(MouseEvent arg0)  
  {  
   repaint();  
   if (this.drag_status == 1)  
   {  
    this.c3 = arg0.getX();  
    this.c4 = arg0.getY();  
    try  
    {  
     draggedScreen();  
    }  
    catch (Exception e)  
    {  
     e.printStackTrace();  
    }  
   }  
  }  
  public void mouseDragged(MouseEvent arg0)  
  {  
   repaint();  
   this.drag_status = 1;  
   this.c3 = arg0.getX();  
   this.c4 = arg0.getY();  
  }  
  public void mouseMoved(MouseEvent arg0) {}  
  public void paint(Graphics g)  
  {  
   super.paint(g);  
   int w = this.c1 - this.c3;  
   int h = this.c2 - this.c4;  
   w *= -1;  
   h *= -1;  
   if (w < 0) {  
    w *= -1;  
   }  
   g.drawRect(this.c1, this.c2, w, h);  
  }  
 }  

ImagePanel.java
 package com.cooltrickshome;  
 import java.awt.Dimension;  
 import java.awt.Graphics;  
 import java.awt.Image;  
 import javax.swing.ImageIcon;  
 import javax.swing.JPanel;  
 class ImagePanel  
  extends JPanel  
 {  
  private Image img;  
  public ImagePanel(String img)  
  {  
   this(new ImageIcon(img).getImage());  
  }  
  public ImagePanel(Image img)  
  {  
   this.img = img;  
   Dimension size = new Dimension(img.getWidth(null), img.getHeight(null));  
   setPreferredSize(size);  
   setMinimumSize(size);  
   setMaximumSize(size);  
   setSize(size);  
   setLayout(null);  
  }  
  public void paintComponent(Graphics g)  
  {  
   g.drawImage(this.img, 0, 0, null);  
  }  
 }  

ImageResolver.dll
https://github.com/csanuragjain/extra/blob/master/ImageResolver/Software/ImageResolver.dll?raw=true

Hope it helps :)

3 comments:

  1. Thanks for sharing this post. I'm very interested in this topic.
    onlineconvertfree.com

    ReplyDelete
  2. Montre Rolex Suisse Pas Cher, combining elegant style and cutting-edge technology, a variety of styles of Montre omega Suisse Pas Cher, the pointer walks between your exclusive taste style.

    ReplyDelete
  3. The third-party PDF picture extractor purposes are designed to extract all the photographs from the chosen PDF file(s). There are a number of PDF Picture Extractors accessible available in the market. We are able to simply and safely extract a file in far much less time through the use of any certainly one of them. If you want to learn more about this topic please visit Website

    ReplyDelete