ConnectMP - Taking Multi-Process Data Sharing to the moon
Contribute ยท Community ยท Documentation
๐ซ
Introduction :
ConnectMP is the easiest and efficient way to share data between Processes. It's superfast, can handle big datas, can create multiple data connection and really simple to get started.
ConnectMP is created out of pure Frustration of not being able to find a good solution to comminucate between Processes
๐ฅ
Installation :
via pip (recommended) :
pip install connectmp
๐ง
Quickstart : (Docs)
โ Please read till the end to find what you're looking for!
๐ค
connectmp.Process :
from connectmp import Process
This Process class is same as multiprocessing.Process, it's just modified. So, just use connectmp.Process where you would use multiprocessing.Process
import time
from connectmp import Process
def do_something(connection):
connection.data = "Sending Some Data."
def track_data(connection):
time.sleep(1)
print(f"Track i: {connection.data}")
if __name__ == '__main__':
p1 = Process(target=do_something, connection=True)
p2 = Process(target=track_data, args=(p1.connection,))
p1.start()
p2.start()
Just let me explain what's happening here. we have 2 functions, one is do_something which just sends data and track_data which recieves the data and print's it. You can see in p1 we are not pasing any argument but our function do_something requires a connection argument.
When you enable Process(connection=True) Your target function will recieve a keyword-argument value named connection which's value will contain the Connection object. This Connection object is an individual connection. But You can of course share this connection with other process.
in p2 we shared the connection of our p1 by p1.connection which returns the connection object of p1. In track_data function we just recieve the data from p1.connection. You can send and recieve data from and to each other.
And yeah to send data all you do is assign the data to connection.data and also to get data you have to do it with connection.data
โ
HOLD ON! You might also be looking for something like this down below!
So, if you're wondering if we can create Conenction seperately without creting Process object?
๐ฅจ
connectmp.Connection
from connectmp import Connection
You can create your own Connection with this. Let me show you how:
from connectmp import Connection
connection = Connection()
that's it! Connection! To use it just do it like how we did before, pass it as an argument and use it anywhere you want
โ NOTE: Make sureProcess(connection=True)is not enabled, or it will conflict with your function params. turnProcess(connection=True)when you want to create anotherConnectionand get it in the function as an argument('connection').
๐ฎ
Example:
import time
from connectmp import Process, Connection
def do_something(connection):
connection.data = "Sending Some Data."
def track_data(connection):
time.sleep(1)
print(f"Track i: {connection.data}")
if __name__ == '__main__':
conn = Connection() # Creating connection
p1 = Process(target=do_something, args=(conn,))
p2 = Process(target=track_data, args=(conn,))
p1.start()
p2.start()
Now each Process can communicate with each other! That's all you need to know about ConnectMP. Hope this helped you
3 Aug 6, 2022
24 Aug 28, 2022
3 Dec 1, 2022
6 Jun 8, 2022
4 Dec 8, 2021
37 Nov 2, 2022
1.5k Dec 31, 2022
12 Jan 2, 2023
29 Aug 28, 2022
1 Dec 12, 2021
41 Dec 16, 2022
5 Nov 09, 2021
8 Nov 27, 2022
0 Jun 21, 2022
2 Jan 20, 2022
39 Dec 23, 2022
8 Oct 31, 2021
1 Dec 27, 2021
3 Jun 06, 2021
1 Oct 11, 2022
8 Jan 05, 2023
2 Sep 03, 2022
638 Dec 13, 2022
4 Jan 26, 2022
20 Dec 20, 2022
6 Sep 09, 2021
0 Sep 05, 2021
2 Jan 29, 2022