Re: Timer/Events or Socket

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Timer/Events or Socket

Patrick Moore
I am interested in creating a link between 3D Slicer and Blender (or any software for that matter) that allows models in the Slicer Scene to track or be updated with information from Blender.

I have read the scripted modules tutorial and browsed some basic modules form the repository.  I have a basic understanding of creating a widget that performs an operation on a button click.  I have been able to import and transform models via scripting and set their surface color.  I have written a simple module which checks a temp directory for for .ply files and imports them if they don't exist in the Slicer scene, it will then read an .xml file and update the transform matrices accordingly.  I have an equivalent operator in Blender which exports the models/transforms etc from the Blender scene.


I would now like to have this update in a live or semi live way instead of having to click each time to update.  I envision Blender and Slicer side by side allowing the user to model/transform items in Blender and view the volume data in Slicer simultaneously.  In Blender we have timers and handlers where code can be executed periodically.

1).  Are there timers, events or event handlers in Slicer that can be accessed through the python API?

2)  Are there any existing modules that use sockets or timers that I can read/explore?

Thanks for any help.

Sincerely,
Patrick


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ
Reply | Threaded
Open this post in threaded view
|

Re: Timer/Events or Socket

Andras Lasso-2

OpenIGTLink module in Slicer can send/receive most node types through a very simple socket -based protocol.

 

Probably the simplest is to just use this, as everything is already fully implemented and tested on Slicer side.

 

The specification is available at www.openigtlink.org. There are some Python implementations of OpenIGTLink that might be usable.

 

Andras

 

From: [hidden email]
Sent: Sunday, April 2, 2017 12:02 PM
To: [hidden email]
Subject: Re: [slicer-users] Timer/Events or Socket

 

I am interested in creating a link between 3D Slicer and Blender (or any software for that matter) that allows models in the Slicer Scene to track or be updated with information from Blender.

I have read the scripted modules tutorial and browsed some basic modules form the repository.  I have a basic understanding of creating a widget that performs an operation on a button click.  I have been able to import and transform models via scripting and set their surface color.  I have written a simple module which checks a temp directory for for .ply files and imports them if they don't exist in the Slicer scene, it will then read an .xml file and update the transform matrices accordingly.  I have an equivalent operator in Blender which exports the models/transforms etc from the Blender scene.


I would now like to have this update in a live or semi live way instead of having to click each time to update.  I envision Blender and Slicer side by side allowing the user to model/transform items in Blender and view the volume data in Slicer simultaneously.  In Blender we have timers and handlers where code can be executed periodically.

1).  Are there timers, events or event handlers in Slicer that can be accessed through the python API?

2)  Are there any existing modules that use sockets or timers that I can read/explore?

Thanks for any help.

Sincerely,
Patrick


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ
Reply | Threaded
Open this post in threaded view
|

Re: Timer/Events or Socket

Steve Pieper-2
Hi Patrick - 

That looks really cool - I have a couple ideas for you. 

Andras is right that OpenIGTLink is already set up in Slicer, but I don't know of any stand alone python implementations that would be easy to use in Blender.  The protocol's not overly complex but you'd need to write some socket and other translation code.

Based on what you have so far, maybe the easiest is to use a QFileSystemWatcher [1].  You could add your temp path and then you can connect to the directoryChanged signal and update the models after they are modified by Blender.  This is tied into the Qt event loop so it should be pretty seamless on the Slicer side.  Not sure if there's any comparable behavior in Blender's UI toolkit

Another possibility is do a direct socket connection between the two programs so you can send arbitrary messages.  The QSocketNotifier [2] can also be used to integrate socket events with the Qt event loop.  Again you'd need to write some socket code on the Blender side, but it could be simple if you just want to send short messages when files are ready to be read by the other program.

I'll also throw out that I've been working for a while on embedding web server functionality in Slicer [3].  It uses the QSocketNotifier and works well.  I have it set up so that some data types (e.g. images, volumes) can be transferred directly via http rather than going through intermediate files.  Transferring models is still a work in progress, but lately I'm looking at glTF [5] as an interesting option.  Again I'm not sure how that would work on the Blender side.

Hope that helps - it will be great to see more Slicer/Blender synergy.  Several of us had discussed options in the past but nobody knew Blender well enough to make it happen.

Best,
Steve







On Sun, Apr 2, 2017 at 12:21 PM, Andras Lasso <[hidden email]> wrote:

OpenIGTLink module in Slicer can send/receive most node types through a very simple socket -based protocol.

 

Probably the simplest is to just use this, as everything is already fully implemented and tested on Slicer side.

 

The specification is available at www.openigtlink.org. There are some Python implementations of OpenIGTLink that might be usable.

 

Andras

 

From: [hidden email]
Sent: Sunday, April 2, 2017 12:02 PM
To: [hidden email]
Subject: Re: [slicer-users] Timer/Events or Socket

 

I am interested in creating a link between 3D Slicer and Blender (or any software for that matter) that allows models in the Slicer Scene to track or be updated with information from Blender.

I have read the scripted modules tutorial and browsed some basic modules form the repository.  I have a basic understanding of creating a widget that performs an operation on a button click.  I have been able to import and transform models via scripting and set their surface color.  I have written a simple module which checks a temp directory for for .ply files and imports them if they don't exist in the Slicer scene, it will then read an .xml file and update the transform matrices accordingly.  I have an equivalent operator in Blender which exports the models/transforms etc from the Blender scene.


I would now like to have this update in a live or semi live way instead of having to click each time to update.  I envision Blender and Slicer side by side allowing the user to model/transform items in Blender and view the volume data in Slicer simultaneously.  In Blender we have timers and handlers where code can be executed periodically.

1).  Are there timers, events or event handlers in Slicer that can be accessed through the python API?

2)  Are there any existing modules that use sockets or timers that I can read/explore?

Thanks for any help.

Sincerely,
Patrick


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ
Reply | Threaded
Open this post in threaded view
|

Re: Timer/Events or Socket

Andras Lasso-2
In reply to this post by Andras Lasso-2

There are multiple Python OpenIGTLink implementations that should be easy use from Blender. I haven’t tried them but they seem to nice and already available through packaging managers. For example:

https://pypi.python.org/pypi/pyIGTLink/0.2.1

https://libraries.io/github/nenetto/OpenIGTLink

 

The advantage of OpenIGTLink is that it is very simple (difficult to define anything simpler over raw sockets) and very fast. However, it is only used in the image-guided therapy domain.

 

It would be nice if Slicer supported some more general interfaces as well. Does Blender already have interface for data exchange with external applications? What protocol does that use? Maybe we could add support for that protocol in Slicer.

 

I’ve been thinking about adding similar bridge to free, Python-based CAD software, such as FreeCAD.

 

Andras

 

From: Steve Pieper [mailto:[hidden email]]
Sent: April 2, 2017 14:42
To: Andras Lasso <[hidden email]>
Cc: Patrick Moore <[hidden email]>; SPL Slicer Users <[hidden email]>
Subject: Re: [slicer-users] Timer/Events or Socket

 

Hi Patrick - 

 

That looks really cool - I have a couple ideas for you. 

 

Andras is right that OpenIGTLink is already set up in Slicer, but I don't know of any stand alone python implementations that would be easy to use in Blender.  The protocol's not overly complex but you'd need to write some socket and other translation code.

 

Based on what you have so far, maybe the easiest is to use a QFileSystemWatcher [1].  You could add your temp path and then you can connect to the directoryChanged signal and update the models after they are modified by Blender.  This is tied into the Qt event loop so it should be pretty seamless on the Slicer side.  Not sure if there's any comparable behavior in Blender's UI toolkit

 

Another possibility is do a direct socket connection between the two programs so you can send arbitrary messages.  The QSocketNotifier [2] can also be used to integrate socket events with the Qt event loop.  Again you'd need to write some socket code on the Blender side, but it could be simple if you just want to send short messages when files are ready to be read by the other program.

 

I'll also throw out that I've been working for a while on embedding web server functionality in Slicer [3].  It uses the QSocketNotifier and works well.  I have it set up so that some data types (e.g. images, volumes) can be transferred directly via http rather than going through intermediate files.  Transferring models is still a work in progress, but lately I'm looking at glTF [5] as an interesting option.  Again I'm not sure how that would work on the Blender side.

 

Hope that helps - it will be great to see more Slicer/Blender synergy.  Several of us had discussed options in the past but nobody knew Blender well enough to make it happen.

 

Best,

Steve

 

 

 

 

 

 

 

On Sun, Apr 2, 2017 at 12:21 PM, Andras Lasso <[hidden email]> wrote:

OpenIGTLink module in Slicer can send/receive most node types through a very simple socket -based protocol.

 

Probably the simplest is to just use this, as everything is already fully implemented and tested on Slicer side.

 

The specification is available at www.openigtlink.org. There are some Python implementations of OpenIGTLink that might be usable.

 

Andras

 

From: [hidden email]
Sent: Sunday, April 2, 2017 12:02 PM
To: [hidden email]
Subject: Re: [slicer-users] Timer/Events or Socket

 

I am interested in creating a link between 3D Slicer and Blender (or any software for that matter) that allows models in the Slicer Scene to track or be updated with information from Blender.

 

I have read the scripted modules tutorial and browsed some basic modules form the repository.  I have a basic understanding of creating a widget that performs an operation on a button click.  I have been able to import and transform models via scripting and set their surface color.  I have written a simple module which checks a temp directory for for .ply files and imports them if they don't exist in the Slicer scene, it will then read an .xml file and update the transform matrices accordingly.  I have an equivalent operator in Blender which exports the models/transforms etc from the Blender scene.

 

 

I would now like to have this update in a live or semi live way instead of having to click each time to update.  I envision Blender and Slicer side by side allowing the user to model/transform items in Blender and view the volume data in Slicer simultaneously.  In Blender we have timers and handlers where code can be executed periodically.

 

1).  Are there timers, events or event handlers in Slicer that can be accessed through the python API?

 

2)  Are there any existing modules that use sockets or timers that I can read/explore?

 

Thanks for any help.

 

Sincerely,

Patrick

 


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ

 


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ
Reply | Threaded
Open this post in threaded view
|

Re: Timer/Events or Socket

Patrick Moore
In reply to this post by Andras Lasso-2
1.  Thank you for the IGT links.  I will have to poke around.

2.  To my knowledge Blender does not have any standard interface for data exchange.  But, there are a few ways it has been discussed

The two areas that I can think of are
-General python sockets
-Network rendering, allowing network work stations to render different parts of a project
-Game development.  There are lots of examples of network/socket data exchange for games.
-One off projects with use serial ports or sockets to control objects in the 3d view.
-An addon which uses a "Verse" server.  https://github.com/verse/verse-blender


3  In terms of creating some kind of bridge, I know the Blender API well enough to get us pointed in the right direction and I know some people to ask around for areas where we may struggle.  Blender has a pretty good track record of if a concept is well proven with a python addon and demonstrated demand in the user base, then they often pick it up in the core code part of blender.  I was part of the PieMenus python addon before blender core developers picked it up and implemented it.  Having a bridge between the two software's could really expand the user base of both.

Best,
Patrick

On Sun, Apr 2, 2017 at 3:38 PM, Andras Lasso <[hidden email]> wrote:

There are multiple Python OpenIGTLink implementations that should be easy use from Blender. I haven’t tried them but they seem to nice and already available through packaging managers. For example:

https://pypi.python.org/pypi/pyIGTLink/0.2.1

https://libraries.io/github/nenetto/OpenIGTLink

 

The advantage of OpenIGTLink is that it is very simple (difficult to define anything simpler over raw sockets) and very fast. However, it is only used in the image-guided therapy domain.

 

It would be nice if Slicer supported some more general interfaces as well. Does Blender already have interface for data exchange with external applications? What protocol does that use? Maybe we could add support for that protocol in Slicer.

 

I’ve been thinking about adding similar bridge to free, Python-based CAD software, such as FreeCAD.

 

Andras

 

From: Steve Pieper [mailto:[hidden email]]
Sent: April 2, 2017 14:42
To: Andras Lasso <[hidden email]>
Cc: Patrick Moore <[hidden email]>; SPL Slicer Users <[hidden email]>


Subject: Re: [slicer-users] Timer/Events or Socket

 

Hi Patrick - 

 

That looks really cool - I have a couple ideas for you. 

 

Andras is right that OpenIGTLink is already set up in Slicer, but I don't know of any stand alone python implementations that would be easy to use in Blender.  The protocol's not overly complex but you'd need to write some socket and other translation code.

 

Based on what you have so far, maybe the easiest is to use a QFileSystemWatcher [1].  You could add your temp path and then you can connect to the directoryChanged signal and update the models after they are modified by Blender.  This is tied into the Qt event loop so it should be pretty seamless on the Slicer side.  Not sure if there's any comparable behavior in Blender's UI toolkit

 

Another possibility is do a direct socket connection between the two programs so you can send arbitrary messages.  The QSocketNotifier [2] can also be used to integrate socket events with the Qt event loop.  Again you'd need to write some socket code on the Blender side, but it could be simple if you just want to send short messages when files are ready to be read by the other program.

 

I'll also throw out that I've been working for a while on embedding web server functionality in Slicer [3].  It uses the QSocketNotifier and works well.  I have it set up so that some data types (e.g. images, volumes) can be transferred directly via http rather than going through intermediate files.  Transferring models is still a work in progress, but lately I'm looking at glTF [5] as an interesting option.  Again I'm not sure how that would work on the Blender side.

 

Hope that helps - it will be great to see more Slicer/Blender synergy.  Several of us had discussed options in the past but nobody knew Blender well enough to make it happen.

 

Best,

Steve

 

 

 

 

 

 

 

On Sun, Apr 2, 2017 at 12:21 PM, Andras Lasso <[hidden email]> wrote:

OpenIGTLink module in Slicer can send/receive most node types through a very simple socket -based protocol.

 

Probably the simplest is to just use this, as everything is already fully implemented and tested on Slicer side.

 

The specification is available at www.openigtlink.org. There are some Python implementations of OpenIGTLink that might be usable.

 

Andras

 

From: [hidden email]
Sent: Sunday, April 2, 2017 12:02 PM
To: [hidden email]
Subject: Re: [slicer-users] Timer/Events or Socket

 

I am interested in creating a link between 3D Slicer and Blender (or any software for that matter) that allows models in the Slicer Scene to track or be updated with information from Blender.

 

I have read the scripted modules tutorial and browsed some basic modules form the repository.  I have a basic understanding of creating a widget that performs an operation on a button click.  I have been able to import and transform models via scripting and set their surface color.  I have written a simple module which checks a temp directory for for .ply files and imports them if they don't exist in the Slicer scene, it will then read an .xml file and update the transform matrices accordingly.  I have an equivalent operator in Blender which exports the models/transforms etc from the Blender scene.

 

 

I would now like to have this update in a live or semi live way instead of having to click each time to update.  I envision Blender and Slicer side by side allowing the user to model/transform items in Blender and view the volume data in Slicer simultaneously.  In Blender we have timers and handlers where code can be executed periodically.

 

1).  Are there timers, events or event handlers in Slicer that can be accessed through the python API?

 

2)  Are there any existing modules that use sockets or timers that I can read/explore?

 

Thanks for any help.

 

Sincerely,

Patrick

 


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ

 



_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ
Reply | Threaded
Open this post in threaded view
|

Re: Timer/Events or Socket

Patrick Moore
In reply to this post by Andras Lasso-2
Dear Slicer Users,

I have made good progress using the QFileSystemWatcher and writing files to a temporary directory and then hacking together some code on the Slicer and Blender end to manage their communication.  It's functional and definitely meets my needs.  I think that sockets may be the next step for a more robust interface but certainly QFileSystemWatcher has been a good proof of concept. Here it is in action.


Sincerely,
Patrick

On Sun, Apr 2, 2017 at 4:55 PM, Patrick Moore <[hidden email]> wrote:
1.  Thank you for the IGT links.  I will have to poke around.

2.  To my knowledge Blender does not have any standard interface for data exchange.  But, there are a few ways it has been discussed

The two areas that I can think of are
-General python sockets
-Network rendering, allowing network work stations to render different parts of a project
-Game development.  There are lots of examples of network/socket data exchange for games.
-One off projects with use serial ports or sockets to control objects in the 3d view.
-An addon which uses a "Verse" server.  https://github.com/verse/verse-blender


3  In terms of creating some kind of bridge, I know the Blender API well enough to get us pointed in the right direction and I know some people to ask around for areas where we may struggle.  Blender has a pretty good track record of if a concept is well proven with a python addon and demonstrated demand in the user base, then they often pick it up in the core code part of blender.  I was part of the PieMenus python addon before blender core developers picked it up and implemented it.  Having a bridge between the two software's could really expand the user base of both.

Best,
Patrick


On Sun, Apr 2, 2017 at 3:38 PM, Andras Lasso <[hidden email]> wrote:

There are multiple Python OpenIGTLink implementations that should be easy use from Blender. I haven’t tried them but they seem to nice and already available through packaging managers. For example:

https://pypi.python.org/pypi/pyIGTLink/0.2.1

https://libraries.io/github/nenetto/OpenIGTLink

 

The advantage of OpenIGTLink is that it is very simple (difficult to define anything simpler over raw sockets) and very fast. However, it is only used in the image-guided therapy domain.

 

It would be nice if Slicer supported some more general interfaces as well. Does Blender already have interface for data exchange with external applications? What protocol does that use? Maybe we could add support for that protocol in Slicer.

 

I’ve been thinking about adding similar bridge to free, Python-based CAD software, such as FreeCAD.

 

Andras

 

From: Steve Pieper [mailto:[hidden email]]
Sent: April 2, 2017 14:42
To: Andras Lasso <[hidden email]>
Cc: Patrick Moore <[hidden email]>; SPL Slicer Users <[hidden email]>


Subject: Re: [slicer-users] Timer/Events or Socket

 

Hi Patrick - 

 

That looks really cool - I have a couple ideas for you. 

 

Andras is right that OpenIGTLink is already set up in Slicer, but I don't know of any stand alone python implementations that would be easy to use in Blender.  The protocol's not overly complex but you'd need to write some socket and other translation code.

 

Based on what you have so far, maybe the easiest is to use a QFileSystemWatcher [1].  You could add your temp path and then you can connect to the directoryChanged signal and update the models after they are modified by Blender.  This is tied into the Qt event loop so it should be pretty seamless on the Slicer side.  Not sure if there's any comparable behavior in Blender's UI toolkit

 

Another possibility is do a direct socket connection between the two programs so you can send arbitrary messages.  The QSocketNotifier [2] can also be used to integrate socket events with the Qt event loop.  Again you'd need to write some socket code on the Blender side, but it could be simple if you just want to send short messages when files are ready to be read by the other program.

 

I'll also throw out that I've been working for a while on embedding web server functionality in Slicer [3].  It uses the QSocketNotifier and works well.  I have it set up so that some data types (e.g. images, volumes) can be transferred directly via http rather than going through intermediate files.  Transferring models is still a work in progress, but lately I'm looking at glTF [5] as an interesting option.  Again I'm not sure how that would work on the Blender side.

 

Hope that helps - it will be great to see more Slicer/Blender synergy.  Several of us had discussed options in the past but nobody knew Blender well enough to make it happen.

 

Best,

Steve

 

 

 

 

 

 

 

On Sun, Apr 2, 2017 at 12:21 PM, Andras Lasso <[hidden email]> wrote:

OpenIGTLink module in Slicer can send/receive most node types through a very simple socket -based protocol.

 

Probably the simplest is to just use this, as everything is already fully implemented and tested on Slicer side.

 

The specification is available at www.openigtlink.org. There are some Python implementations of OpenIGTLink that might be usable.

 

Andras

 

From: [hidden email]
Sent: Sunday, April 2, 2017 12:02 PM
To: [hidden email]
Subject: Re: [slicer-users] Timer/Events or Socket

 

I am interested in creating a link between 3D Slicer and Blender (or any software for that matter) that allows models in the Slicer Scene to track or be updated with information from Blender.

 

I have read the scripted modules tutorial and browsed some basic modules form the repository.  I have a basic understanding of creating a widget that performs an operation on a button click.  I have been able to import and transform models via scripting and set their surface color.  I have written a simple module which checks a temp directory for for .ply files and imports them if they don't exist in the Slicer scene, it will then read an .xml file and update the transform matrices accordingly.  I have an equivalent operator in Blender which exports the models/transforms etc from the Blender scene.

 

 

I would now like to have this update in a live or semi live way instead of having to click each time to update.  I envision Blender and Slicer side by side allowing the user to model/transform items in Blender and view the volume data in Slicer simultaneously.  In Blender we have timers and handlers where code can be executed periodically.

 

1).  Are there timers, events or event handlers in Slicer that can be accessed through the python API?

 

2)  Are there any existing modules that use sockets or timers that I can read/explore?

 

Thanks for any help.

 

Sincerely,

Patrick

 


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ

 




_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ
Reply | Threaded
Open this post in threaded view
|

Re: Timer/Events or Socket

Andras Lasso-2
In reply to this post by Andras Lasso-2

Hi Patrick,

 

Thanks for the update, the data bridge seems to work very nicely for pushing surface meshes from Blender to Slicer!

 

What use cases do you have in mind?

 

For me dental implant planning is not an obvious application, as you can already do it pretty well in Slicer: you can import medical images, segment structures, register with other images as needed, import device models, plan their position and inspect the plan both in 2D and 3D. Slicer can also connect to surgical navigation systems and robotic devices to guide the implant procedure. There have been a number of projects in this field, for example a similar screw placement training planning and evaluation: https://www.youtube.com/watch?v=XN3Tp8jaYdQ.

 

Blender is superior in mesh editing, surface rendering quality, and animation capabilities. To take advantage of these features, data push from Blender would not be enough, but we would also need data push and pull from Slicer. Do you have solution in mind for these?

 

Andras

 

From: Patrick Moore [mailto:[hidden email]]
Sent: April 16, 2017 19:05
To: Andras Lasso <[hidden email]>
Cc: Steve Pieper <[hidden email]>; SPL Slicer Users <[hidden email]>
Subject: Re: [slicer-users] Timer/Events or Socket

 

Dear Slicer Users,

 

I have made good progress using the QFileSystemWatcher and writing files to a temporary directory and then hacking together some code on the Slicer and Blender end to manage their communication.  It's functional and definitely meets my needs.  I think that sockets may be the next step for a more robust interface but certainly QFileSystemWatcher has been a good proof of concept. Here it is in action.

 

 

Sincerely,

Patrick

 

On Sun, Apr 2, 2017 at 4:55 PM, Patrick Moore <[hidden email]> wrote:

1.  Thank you for the IGT links.  I will have to poke around.

 

2.  To my knowledge Blender does not have any standard interface for data exchange.  But, there are a few ways it has been discussed

 

The two areas that I can think of are

-General python sockets

-Network rendering, allowing network work stations to render different parts of a project

-Game development.  There are lots of examples of network/socket data exchange for games.

-One off projects with use serial ports or sockets to control objects in the 3d view.

-An addon which uses a "Verse" server.  https://github.com/verse/verse-blender

 

 

3  In terms of creating some kind of bridge, I know the Blender API well enough to get us pointed in the right direction and I know some people to ask around for areas where we may struggle.  Blender has a pretty good track record of if a concept is well proven with a python addon and demonstrated demand in the user base, then they often pick it up in the core code part of blender.  I was part of the PieMenus python addon before blender core developers picked it up and implemented it.  Having a bridge between the two software's could really expand the user base of both.

 

Best,

Patrick

 

 

On Sun, Apr 2, 2017 at 3:38 PM, Andras Lasso <[hidden email]> wrote:

There are multiple Python OpenIGTLink implementations that should be easy use from Blender. I haven’t tried them but they seem to nice and already available through packaging managers. For example:

https://pypi.python.org/pypi/pyIGTLink/0.2.1

https://libraries.io/github/nenetto/OpenIGTLink

 

The advantage of OpenIGTLink is that it is very simple (difficult to define anything simpler over raw sockets) and very fast. However, it is only used in the image-guided therapy domain.

 

It would be nice if Slicer supported some more general interfaces as well. Does Blender already have interface for data exchange with external applications? What protocol does that use? Maybe we could add support for that protocol in Slicer.

 

I’ve been thinking about adding similar bridge to free, Python-based CAD software, such as FreeCAD.

 

Andras

 

From: Steve Pieper [mailto:[hidden email]]
Sent: April 2, 2017 14:42
To: Andras Lasso <
[hidden email]>
Cc: Patrick Moore <
[hidden email]>; SPL Slicer Users <[hidden email]>


Subject: Re: [slicer-users] Timer/Events or Socket

 

Hi Patrick - 

 

That looks really cool - I have a couple ideas for you. 

 

Andras is right that OpenIGTLink is already set up in Slicer, but I don't know of any stand alone python implementations that would be easy to use in Blender.  The protocol's not overly complex but you'd need to write some socket and other translation code.

 

Based on what you have so far, maybe the easiest is to use a QFileSystemWatcher [1].  You could add your temp path and then you can connect to the directoryChanged signal and update the models after they are modified by Blender.  This is tied into the Qt event loop so it should be pretty seamless on the Slicer side.  Not sure if there's any comparable behavior in Blender's UI toolkit

 

Another possibility is do a direct socket connection between the two programs so you can send arbitrary messages.  The QSocketNotifier [2] can also be used to integrate socket events with the Qt event loop.  Again you'd need to write some socket code on the Blender side, but it could be simple if you just want to send short messages when files are ready to be read by the other program.

 

I'll also throw out that I've been working for a while on embedding web server functionality in Slicer [3].  It uses the QSocketNotifier and works well.  I have it set up so that some data types (e.g. images, volumes) can be transferred directly via http rather than going through intermediate files.  Transferring models is still a work in progress, but lately I'm looking at glTF [5] as an interesting option.  Again I'm not sure how that would work on the Blender side.

 

Hope that helps - it will be great to see more Slicer/Blender synergy.  Several of us had discussed options in the past but nobody knew Blender well enough to make it happen.

 

Best,

Steve

 

 

 

 

 

 

 

On Sun, Apr 2, 2017 at 12:21 PM, Andras Lasso <[hidden email]> wrote:

OpenIGTLink module in Slicer can send/receive most node types through a very simple socket -based protocol.

 

Probably the simplest is to just use this, as everything is already fully implemented and tested on Slicer side.

 

The specification is available at www.openigtlink.org. There are some Python implementations of OpenIGTLink that might be usable.

 

Andras

 

From: [hidden email]
Sent: Sunday, April 2, 2017 12:02 PM
To: [hidden email]
Subject: Re: [slicer-users] Timer/Events or Socket

 

I am interested in creating a link between 3D Slicer and Blender (or any software for that matter) that allows models in the Slicer Scene to track or be updated with information from Blender.

 

I have read the scripted modules tutorial and browsed some basic modules form the repository.  I have a basic understanding of creating a widget that performs an operation on a button click.  I have been able to import and transform models via scripting and set their surface color.  I have written a simple module which checks a temp directory for for .ply files and imports them if they don't exist in the Slicer scene, it will then read an .xml file and update the transform matrices accordingly.  I have an equivalent operator in Blender which exports the models/transforms etc from the Blender scene.

 

 

I would now like to have this update in a live or semi live way instead of having to click each time to update.  I envision Blender and Slicer side by side allowing the user to model/transform items in Blender and view the volume data in Slicer simultaneously.  In Blender we have timers and handlers where code can be executed periodically.

 

1).  Are there timers, events or event handlers in Slicer that can be accessed through the python API?

 

2)  Are there any existing modules that use sockets or timers that I can read/explore?

 

Thanks for any help.

 

Sincerely,

Patrick

 


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ

 

 

 


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ
Reply | Threaded
Open this post in threaded view
|

Re: Timer/Events or Socket

Patrick Moore
In reply to this post by Andras Lasso-2
Thanks Andras

Use cases:

-For dental implants,  the planning is definitely accomplished best in Slicer, however designing a surgical guide based on those positions is where I plan to use Blender.  There are guided surgery systems that essentially use a series of jigs to rigidly guide the drill and implant placement.

-Animation 
-Sending object transforms to Slicer to evaluate orthodontic movements to visualize/anticipate  tooth root positions relative to the bony boundaries that enclose the teeth
-From Slicer:taking objects or tracking data from slicer to take advantage of Blenders rendering capabilities.


I think using a tmp directory to handle one time import/export of objects and then a socket connection to communicate what to do with them or to apply changes would handle a large set of generic uses between the 2 softwares.

If live editing is not critical, a user controlled update/import/export would be slower but easier to implement in both directions to and from each software.

Where can I find info on how the Slicer mesh/surface "models" are stored (like verts, edges, faces).


Best,
Patrick

On Apr 17, 2017 12:07 AM, "Andras Lasso" <[hidden email]> wrote:

Hi Patrick,

 

Thanks for the update, the data bridge seems to work very nicely for pushing surface meshes from Blender to Slicer!

 

What use cases do you have in mind?

 

For me dental implant planning is not an obvious application, as you can already do it pretty well in Slicer: you can import medical images, segment structures, register with other images as needed, import device models, plan their position and inspect the plan both in 2D and 3D. Slicer can also connect to surgical navigation systems and robotic devices to guide the implant procedure. There have been a number of projects in this field, for example a similar screw placement training planning and evaluation: https://www.youtube.com/watch?v=XN3Tp8jaYdQ.

 

Blender is superior in mesh editing, surface rendering quality, and animation capabilities. To take advantage of these features, data push from Blender would not be enough, but we would also need data push and pull from Slicer. Do you have solution in mind for 


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ
Reply | Threaded
Open this post in threaded view
|

Re: Timer/Events or Socket

Andras Lasso-2
In reply to this post by Andras Lasso-2

> designing a surgical guide

 

It's a good point. Unfortunately VTK’s Boolean mesh operations implementation is not robust (the result often has random artifacts). We work around that by creating surgical guides, molds, etc. using segmentation nodes with binary labelmap as master representation: essentially, we edit segments as a labelmap volumes and generate closed surface mesh representation on-the-fly for rendering in 3D and for 3D printing. See for example this tutorial:

https://www.slicer.org/wiki/Documentation/Nightly/Training#Segmentation_for_3D_printing

 

It works quite well, but in some cases it would be nice to be able to just combine segments represented as closed surfaces with Boolean operations without remeshing.

 

There is some ongoing work in developing Slicer add-ons for dental use, which may be relevant for you - see https://sites.google.com/a/umich.edu/dentistry-image-computing/

 

You may also find the Sequences extension useful, which allows recording, replaying, and editing temporal sequences of any nodes (rigid and deforming transforms meshes, image volumes, landmark points, etc). Unfortunately, editing of Sequnces on the GUI is quite rudimentary and no interpolation between key frames is implemented. It would be nice to somehow use Blender’s animation framework to create 3D+t scenes and import into Slicer.

 

> Where can I find info on how the Slicer mesh/surface "models" are stored (like verts, edges, faces).

 

Slicer uses VTK (www.vtk.org) for all surface mesh operations (reading/writing file, processing, visualization). In-memory storage is usually polygonal surface mesh in vtkPolyData (but Slicer also supports volumetric meshes stored in vtkUnstructuredGrid). Slicer can read/write many mesh formats including stl, ply, obj, vtp, vtk, …, but doesn’t always support all features of the file formats.

 

Command&control through sockets and exchanging data through files may work well. Actually, this is how Slicer’s MatlabBridge works (https://www.slicer.org/wiki/Documentation/Nightly/Extensions/MatlabBridge): Slicer calls remote methods through OpenIGTLink and exchanges most input/output data sets as files. It shouldn't be difficult to implement the MatlabBridge’s server-side component in Blender and then in Slicer you wouldn’t need to modify much, if anything. Let me know if you were interested in exploring this option.

 

Andras

 

From: [hidden email]
Sent: Monday, April 17, 2017 1:29 PM
To: [hidden email]
Cc: [hidden email]; [hidden email]
Subject: RE: [slicer-users] Timer/Events or Socket

 

Thanks Andras

Use cases:

-For dental implants,  the planning is definitely accomplished best in Slicer, however designing a surgical guide based on those positions is where I plan to use Blender.  There are guided surgery systems that essentially use a series of jigs to rigidly guide the drill and implant placement.

-Animation 
-Sending object transforms to Slicer to evaluate orthodontic movements to visualize/anticipate  tooth root positions relative to the bony boundaries that enclose the teeth
-From Slicer:taking objects or tracking data from slicer to take advantage of Blenders rendering capabilities.


I think using a tmp directory to handle one time import/export of objects and then a socket connection to communicate what to do with them or to apply changes would handle a large set of generic uses between the 2 softwares.

If live editing is not critical, a user controlled update/import/export would be slower but easier to implement in both directions to and from each software.

Where can I find info on how the Slicer mesh/surface "models" are stored (like verts, edges, faces).


Best,
Patrick

On Apr 17, 2017 12:07 AM, "Andras Lasso" <[hidden email]> wrote:

Hi Patrick,

 

Thanks for the update, the data bridge seems to work very nicely for pushing surface meshes from Blender to Slicer!

 

What use cases do you have in mind?

 

For me dental implant planning is not an obvious application, as you can already do it pretty well in Slicer: you can import medical images, segment structures, register with other images as needed, import device models, plan their position and inspect the plan both in 2D and 3D. Slicer can also connect to surgical navigation systems and robotic devices to guide the implant procedure. There have been a number of projects in this field, for example a similar screw placement training planning and evaluation: https://www.youtube.com/watch?v=XN3Tp8jaYdQ.

 

Blender is superior in mesh editing, surface rendering quality, and animation capabilities. To take advantage of these features, data push from Blender would not be enough, but we would also need data push and pull from Slicer. Do you have solution in mind for 


_______________________________________________
slicer-users mailing list
[hidden email]
http://massmail.spl.harvard.edu/mailman/listinfo/slicer-users
To unsubscribe: send email to [hidden email] with unsubscribe as the subject
http://www.slicer.org/slicerWiki/index.php/Documentation/4.3/FAQ