Has anyone successfully gotten comskip working?

[williamthrilliam]

I'm following this guide, and it seems to somewhat work. My shows get converted to mkv and have their subtitles extracted, but the commercials are still there. Has anyone gotten this working? Do I need to enable any other plugin or setting? 

Thanks!