Elasticsearch 2.0 released

Released Elasticsearch 2.0. A major milestone and achievement of the whole team, and wonderful contributions from the community. New type of aggregations called pipeline aggs, simplified query DSL by merging query and filter concepts, better compression options, hardened security by enabling security manager, hardening of FS behavior (fsync, more checksums, atomic renames), performance, consistent mapping behavior, and many more. Also, it bundles Lucene 5 release, which includes numerous improvements.

tf–idf - best known weighting scheme in information retrieval

tf–idf, short for term frequency–inverse document frequency.
Best known weighting scheme in information retrieval
Good explanation: https://class.coursera.org/nlp/lecture/187

I have just released Allegro OpenSource: Elasticsearch reindex tool
http://allegrotech.io/elasticsearch-reindex-tool.html

Logstash roadmap

Logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs.

It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.

Now we have also Logstash roadmap

Spring Cloud 1.0.0 Available Now

Spring Cloud provides tools for developers to quickly build some of the common patterns in distributed systems (e.g. configuration management, service discovery, circuit breakers, intelligent routing, micro-proxy, control bus, one-time tokens, global locks, leadership election, distributed sessions, cluster state). Coordination of distributed systems leads to boiler plate patterns, and using Spring Cloud developers can quickly stand up services and applications that implement those patterns. They will work well in any distributed environment, including the developer’s own laptop, bare metal data centres, and managed platforms such as Cloud Foundry.

More info here:
https://spring.io/blog/2015/03/04/spring-cloud-1-0-0-available-nowLokalizacja

Good article about lucene internals

You can read about lucene implementation concerning posting lists and filters here:

http://www.elasticsearch.org/blog/frame-of-reference-and-roaring-bitmaps/

Announcing MongoDB 3.0

Today MongoDB 3.0 is generally available; you can download now.

A tutorial how to add authorisation layer over elasticsearch cluster using Nginx

Read here: http://www.elasticsearch.org/blog/playing-http-tricks-nginx/

Lucene nightly benchmarks

Each night, an automated Python tool checks out the Lucene/Solr trunk source code and runs multiple benchmarks: indexing the entire Wikipedia English export three times (with different settings / document sizes); running a near-real-time latency test; running a set of "hardish" auto-generated queries and tasks. The tests take around 2.5 hours to run, and the results are verified against the previous run and then added to the graphs linked below.
The goal is to spot any long-term regressions (or, gains!) in Lucene's performance that might otherwise accidentally slip past the committers, hopefully avoiding the fate of the boiling frog.

See more details in http://people.apache.org/~mikemccand/lucenebench/

The open source Markdown editor for OS X.

MacDown is an open source Markdown editor for OS X, released under the MIT License.

Look here:
http://macdown.uranusjr.com/

Using SSH Agent

SSH agent allows a user to enter their passphrase(s) for unlocking various SSH keys once at the start of a session to unlock the keys and from then on for the duration of the session the user no longer has to enter the pass phrase(s).
Conventionally setting up SSH agent for use is a bit of a pain as it has to be run before the user session is started.
 

Storing Passphrases in the Keychain

To store the passphrase for your default key in the Keychain open a Terminal and run:

ssh-add -K

And to store the passphrase for a different key run:

ssh-add -K /path/to/private/key/file

When prompted for your passphrase enter it and that is it.

Couchbase Live Europe

Free Couchbase conference in London.

At this event, you will learn how to:

• Leverage NoSQL and Hadoop to build real-time big data solutions
• Develop and deploy large-scale NoSQL applications with Couchbase Server
• Create richer mobile experiences for users with big data mobile solutions

See more here:
http://www.cvent.com/events/couchbase-live-europe-re-platforming-for-big-data-success/event-summary-bf96572b15d841a692ee8a164370384d.aspx

Announcing MongoDB 3.0

The upcoming release will be generally available in early March. Write performance has improved by 7x - 10x with WiredTiger and document-level concurrency control, compression reduces storage needs by up to 80%, and new management software called Ops Manager can reduce operational overhead by up to 95% for most tasks. To learn more, please check out the following resources:
https://www.mongodb.com/blog/post/announcing-mongodb-30

Good overview about Apache Lucene

Watch it: http://2014.berlinbuzzwords.de/session/apache-lucene-4