{"id":1030,"date":"2023-09-02T18:34:43","date_gmt":"2023-09-02T18:34:43","guid":{"rendered":"https:\/\/snersbots.co.uk\/?p=1030"},"modified":"2023-09-11T04:12:17","modified_gmt":"2023-09-11T04:12:17","slug":"photo-consolidation-project-get-a-hash","status":"publish","type":"post","link":"https:\/\/snersbots.co.uk\/index.php\/photo-consolidation-project-get-a-hash\/","title":{"rendered":"Photo Consolidation Project &#8211; Get a Hash"},"content":{"rendered":"\n<p>I decided to get the md5 hash of a file. This is pretty much unique. It ignores dates or file sizes which may vary depending on different systems. But it will identify a file as different if it has been resized. There may be a reason to keep different sizes of files &#8211; if you write a website you should know that very well. But technically these are not duplicates. I would not like to accidentally discard a high-res photo and keep the low-res one.<\/p>\n\n\n\n<!--more-->\n\n\n\n<h2 class=\"wp-block-heading\">The Environment<\/h2>\n\n\n\n<p>For setup I needed to:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Install Visual Studio Code &#8211; jokes, I can&#8217;t survive long without it.<\/li>\n\n\n\n<li>Install Python. Yes I am on Windows so it does not come ready installed. Just head across to <a href=\"https:\/\/www.python.org\/downloads\/\">python.org<\/a><\/li>\n\n\n\n<li>Make a git repo and push it up to <a href=\"https:\/\/github.com\/\">github<\/a>. Yes I have a git repo, if you care enough you won&#8217;t have to try hard to find it.<\/li>\n\n\n\n<li>Boot up ChatGPT &#8211; hey ChatGPT understands Python better than me. I don&#8217;t have to go re-inventing wheels.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Compute MD5<\/h2>\n\n\n\n<p>ChatGPT gave me this code:<\/p>\n\n\n\n<pre class=\"wp-block-code has-background\" style=\"background-color:#e9f1f7\"><code>import hashlib\n\ndef compute_md5(file_path):\n    md5_hash = hashlib.md5()\n    with open(file_path, \"rb\") as f:\n        for chunk in iter(lambda: f.read(4096), b\"\"):\n            md5_hash.update(chunk)\n    return md5_hash.hexdigest()\n<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Code Walkthrough<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Hashlib is a module where the magic happens.<\/li>\n\n\n\n<li>def means &#8216;define&#8217;, and here we are defining a function.<\/li>\n\n\n\n<li>The python &#8216;open&#8217; command syntax is <code>open(<em>file, mode<\/em>)<\/code>.<\/li>\n\n\n\n<li>&#8220;rb&#8221; stands for &#8220;read binary&#8221;. Thanks <a href=\"https:\/\/www.w3schools.com\/python\/ref_func_open.asp\">w3schools<\/a>.<\/li>\n\n\n\n<li>The lambda is f.read(4096)<\/li>\n\n\n\n<li>The iteration is a way to process blocks of input from a file &#8211; example in the <a href=\"https:\/\/docs.python.org\/3\/library\/functions.html#iter\">Python documentation<\/a>.<\/li>\n\n\n\n<li>md5 hash is described on <a href=\"https:\/\/stackoverflow.com\/questions\/3431825\/generating-an-md5-checksum-of-a-file\">Stack Overflow<\/a>.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Is Md5 Unique?<\/h2>\n\n\n\n<p>Well no, but it is pretty much unique, and in this use case it is more than enough.<\/p>\n\n\n\n<p>In the unlikely event that we get two different files coming to the same code &#8211; well the user should be checking them anyway. <\/p>\n","protected":false},"excerpt":{"rendered":"<p>I decided to get the md5 hash of a file. This is pretty much unique. It ignores dates or file sizes which may vary depending on different systems. But it will identify a file as different if it has been resized. There may be a reason to keep different sizes of files &#8211; if you <a class=\"read-more\" href=\"https:\/\/snersbots.co.uk\/index.php\/photo-consolidation-project-get-a-hash\/\">The full story here&#8230;<\/a><\/p>\n","protected":false},"author":1,"featured_media":1038,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[70,8],"tags":[69,71],"class_list":["post-1030","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-desktop-apps","category-software","tag-duplicate-photos","tag-python"],"_links":{"self":[{"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/posts\/1030","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/comments?post=1030"}],"version-history":[{"count":8,"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/posts\/1030\/revisions"}],"predecessor-version":[{"id":1050,"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/posts\/1030\/revisions\/1050"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/media\/1038"}],"wp:attachment":[{"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/media?parent=1030"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/categories?post=1030"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/snersbots.co.uk\/index.php\/wp-json\/wp\/v2\/tags?post=1030"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}