Save data from Instagram takeout to a SQLite database

Overview

instagram-to-sqlite

Save data from a Instagram takeout to a SQLite database.

Mise En Place

git clone https://github.com/gavindsouza/instagram-to-sqlite
pip install -e ./instagram-to-sqlite

This tool only supports JSON data takeouts.

Ricing the potatoes

$ instagram-to-sqlite chats insta-chats.db ~/Downloads/takeout-20190530.zip

This will create a database file called insta-chats.db if one does not already exist.

Serving with the steak

Once you have imported Instagram data into a SQLite database file you can browse your data using Datasette. Install Datasette like so:

$ pip install datasette

Next run

datasette insta-chats.db -o

If you're new to SQL but still want to see what you could do with this, then

  1. Find out what was the first message ever sent on any of your instagram chat rooms*
SELECT
    chat_room "Room", sender_name "Sender", coalesce(content, share, photos, videos, users, audio_files) "Message"
FROM
    chats_messages
GROUP BY
    chat_room
HAVING
    min(timestamp_ms)
ORDER BY
    timestamp_ms

Chat rooms refer to any regular, cross-platform or group chat.

  1. Awhhgee, how about the second messages? A bit unrealistic but still...maybe you really have to KNOW
WITH ordered_messages
     AS (SELECT *,
                Row_number()
                  OVER (
                    partition BY chat_room
                    ORDER BY timestamp_ms) AS 'rank'
         FROM   chats_messages
         )
SELECT
    chat_room "Room", sender_name "Sender", coalesce(content, share, photos, videos, users, audio_files) "Message"
FROM
    ordered_messages
WHERE
    rank = 2
ORDER BY
    timestamp_ms ASC
  1. Okay cool, what if I just want to start reading my chats from their inception like a...normal person...?
SELECT
    type, sender_name, DATETIME(ROUND(timestamp_ms / 1000), 'unixepoch') "Date", coalesce(content, share, photos, videos, users, audio_files) "Message"
FROM
    chats_messages
WHERE
    chat_room = '{chat_room}'
ORDER BY
    timestamp_ms

You will have to figure out the chat_room ID you want to query, but it won't be hard to figure that out.

References

  • Read more about datasette in the docs.

  • Checkout the dogsheep project if you're interested in building your personal data warehouse ;)

Pending stuff

This is the rest of the data available in the Instagram takeout that I haven't built import tools for, yet. Currently, only chat data is covered.

other_data = {
    "login_and_account_creation": [
        "login_activity.json", "logout_activity.json",
        "signup_information.json", "password_change_activity.json",
        "account_privacy_changes.json"
    ],
    "shopping": ["recently_viewed_items.json"],
    "comments": ["post_comments.json", "comments_reported.json"],
    "device_information": ["camera_information.json", "devices.json"],
    "ads_and_content": [
        "suggested_accounts_viewed.json", "ads_viewed.json",
        "posts_viewed.json", "videos_watched.json", "ads_clicked.json"
    ],
    "information_about_you": ["account_based_in.json", "ads_interests.json"],
    "likes": ["liked_posts.json", "liked_comments.json"],
    "content": [
        "posts_1.json", "profile_photos.json", "stories.json",
        "archived_posts.json", "other_content.json",
        "recently_deleted_content.json"
    ],
    "your_topics": [
        "your_reels_topics.json", "your_topics.json",
        "your_reels_sentiments.json"
    ],
    "story_sticker_interactions": [
        "emoji_reactions.json", "quizzes.json", "questions.json",
        "emoji_sliders.json", "polls.json"
    ],
    "comments_settings": ["use_cross-app_messaging.json", "comments_allowed_from.json"],
    "recent_searches": ["tag_searches.json", "account_searches.json"],
    "saved": ["saved_collections.json", "saved_posts.json"],
    "followers_and_following": [
        "removed_suggestions.json", "following_hashtags.json",
        "following.json", "followers.json", "recent_follow_requests.json",
        "pending_follow_requests.json", "close_friends.json"
    ],
    "account_information": [
        "account_information.json", "profile_changes.json",
        "personal_information.json"
    ]
}
Owner
gavin
gavin
A Discord Bot created using Pycord!

Hey, I am Slash Bot. A Bot which works with Slash Commands! Prerequisites Python 3+ Check out. the requirements.txt and install all the pakages. Insta

Saumya Patel 1 Nov 29, 2021
Generate and Visualize Data Lineage from query history

Tokern Lineage Engine Tokern Lineage Engine is fast and easy to use application to collect, visualize and analyze column-level data lineage in databas

Tokern 237 Dec 29, 2022
a translator bot for discord

TranslatorBOT it is a simple and powerful discord bot, it been used for translating includes more than 100 language, it has a lot of integrated comman

Mear. 2 Feb 03, 2022
“ HOLA HUMANS 👋 I'M DAISYX 2.0 ❤️ „ LATEST VERSION OF DAISYX.. Source Code of @Daisyxbot

❤️ DaisyX 2.0 ❤️ A Powerful, Smart And Simple Group Manager ... Written with AioGram , Pyrogram and Telethon... ⭐️ Thanks to everyone who starred Dais

TeamDaisyX 153 Dec 06, 2022
A python script for hitting the kik API to enumerate people based on a username/userlist

kick3d Recon script for enumerating users off of the Kik API. This script has the ability to check single usernames or run through a userlist of usern

Sakura Samurai 19 Oct 04, 2021
Google Translater v2

Google_Translater_V2 Features Supporting 100 More Languages You can Set Your Custom Languages Supporting in Group Configs TG_BOT_TOKEN - Get bot token

Lntechnical 31 Nov 12, 2022
A program that generates discord.py code

discord-py-generator A program that generates discord.py code Setup in cmds.txt file add your user id, client id and bot token you can change the bot

3 Dec 15, 2022
SmsSender v3.0.0 - the script is designed to send free SMS to any number and with any text.

SmsSender v3.0.0 - скрипт предназначен для бесплатной отправки SMS на любой номер и с любым текстом. Возможны небольшие баги, в скором времени исправл

Андрей Сергеев 20 Dec 03, 2021
Instagram - Instagram Account Reporting Tool

Instagram Instagram Account Reporting Tool Installation On Termux $ apt update $

Aryan 6 Nov 18, 2022
Inline Телеграм бот для отправки GIF-изображений из ВКонтакте

VK GIFS Bot VKGIFSBot - удобный бот для отправки GIF-изображений из ВКонтакте в Телеграмe. Работает это очень просто: бот получает токен ВКонтакте API

Sergievsky Nikita 5 Dec 10, 2022
Получение интересной информации о любой пиццерии Додо

dodopizza-abuse Получение инфорации о выбранной пиццерии Додо Установка и запуск на Linux Устанавливаем git и python: apt-get update && apt-get -y ins

Хозя 24 Nov 02, 2022
Automatically pulls specified repository whenever a specified file is pushed. Great for working collaboratively when you need to run something locally.

autopull Simple python tool that allows you to automatically pull from a github repository whenever a file with a specified name is uploaded installat

carreb 0 Sep 27, 2022
Python wrapper library for World Weather Online API

pywwo Python wrapper library for World Weather Online API using lxml.objectify How to use from pywwo import * setKey('your_key', 'free') w=LocalWeat

World Weather Online 20 Dec 19, 2022
A repo containing toolings and software useful for a DevOps Engineer

DevOps-Tooling A repo containing toolings and software useful for a DevOps Engineer (or if you're setting up your Mac from the beginning) Currently se

Mohamed Abukar 45 Dec 12, 2022
Script Crack Facebook, and Instagram 🚶‍♂

in-mbf Script Crack Facebook, and Instagram 🚶‍♂ Bukti Install Script $ pkg update && pkg upgrade $ pkg install git $ pkg install python2 $ pip2 insta

Yumasaa 5 Dec 27, 2021
A discord bot with information and template tracking for pxls.space.

pyCharity A discord bot with information and template tracking for pxls.space. Inspired by Mikarific's Charity bot. Try out the beta version on your s

1 Dec 03, 2021
Wechat-file-cleaner - Clean files in PC WeChat FileStorage directory

Wechat-file-cleaner - Clean files in PC WeChat FileStorage directory

Xingjian Zhang 1 Feb 06, 2022
Migrate BiliBili watched anime to Bangumi

说明 之前为了将B站看过的动画迁移到bangumi写的, 本来只是自己用, 但公开可能对其他人会有帮助. 仓库最近无法维护, 程序有很多缺点, 欢迎 PR 和 Contributors 使用说明 Python版本要求:Python 3.8+ 使用前安装依赖包: pip install -r requ

51 Sep 08, 2022
Running Performance Calculator

Running Performance Calculator 👉 Have you ever wondered if you ran 10km at 2000

Davide Liu 6 Oct 26, 2022
Python API to interact with Uwazi

Python Uwazi API Quick Start To use the API install the requirements pip3 install -r requirements.txt and use it like this: uwazi_adapter = UwaziAdap

HURIDOCS 2 Dec 16, 2021