I'm writing a tool to help with analysis of small-ish logfiles (e.g. 1-2 MB, in rare cases up to 50 MB).
The logfiles come from a file-syncing application, and contain a variety of different loglines:
2016-02-22 21:18:03,872 +0200 INFO pid=670 4664029184:PerfReporter perf_reporter.pyo:71 Current Stats: sync_bo=0, dio=266945, blacklist_len=0, uptime=1601770, pc=60, sync_x=0, sync_y=0, prs=78368, sync_percent=0, corpus=8819, c0=1510, pvm=3095812
2016-02-22 21:18:03,874 +0200 INFO pid=670 4664029184:PerfReporter sync_http_client.pyo:237 Opening direct connection to csi.gstatic.com:443.
2016-02-22 21:19:13,185 +0200 INFO pid=670 4650881024:SyncClientImpressionsThread impression_logger.pyo:278 Heartbeat was added.
2015-06-23 12:15:29,860 +0300 INFO pid=33914 4634906624:Worker-2 snapshot_sqlite.pyo:143 Adding local entry inode=57033344, filename=None
2015-06-23 12:15:29,861 +0300 INFO pid=33914 4634906624:Worker-2 snapshot_sqlite.pyo:171 Adding cloud entry resource_id=file:0B_JGPr4BzMr4dmdCbFBibms5WFk, filename=None
2015-06-23 12:15:29,862 +0300 INFO pid=33914 4634906624:Worker-2 snapshot_sqlite.pyo:253 Updating cloud entry doc_id=0B_JGPr4BzMr4dmdCbFBibms5WFk, filename=~$Foo Bar.xlsx
2015-06-23 12:15:30,247 +0300 INFO pid=33914 4651732992:Batcher batcher.pyo:849 Batcher Stats = file_count = Counter({_COUNT_KEY(direction=_DownloadDirectionType(Direction.DOWNLOAD), action=_FSChangeActionType(Action.CREATE), batch=False, successful=True): 1}), byte_count = Counter({_COUNT_KEY(direction=_DownloadDirectionType(Direction.DOWNLOAD), action=
_FSChangeActionType(Action.CREATE), batch=False, successful=True): 165}), batch_operation_count = Counter(), process_seconds = Counter({_COUNT_KEY(direction=_DownloadDirectionType(Direction.DOWNLOAD), action=_FSChangeActionType(Action.CREATE), batch=False, successful=True): 0.6173379421234131}), duration seconds = 1 (start_time = 1435050929, end_time = 143505093
0)
I'll be parsing out any key-value pairs, as well as several key attributes (e.g. inode number, filename, doc_id)
I would then like a UI (either console or Web UI) that lets you filter by various things, and display the full loglines:
- Filtering by time ranges
- Filtering by inode number, filename, event-type etc.
Are there any existing UIs elements/toolkits/frameworks that allow easy filtering along multiple dimensions?
So for example, you could select an inode number and event-type, and see a full history over time for that combination?
Probably similar to what Splunk/Kibana and ilk allow you to, but available as a stand-alone component? (console or web)
With MASSALYZER you can do it in console. If you need help, ask me!