My experience with couchbase

Draft Disclaimer: Please note that this article is currently in draft form and may undergo revisions before final publication. The content, including information, opinions, and recommendations, is subject to change and may not represent the final version. We appreciate your understanding and patience as we work to refine and improve the quality of this article. Your feedback is valuable in shaping the final release.

Couchbase is not CouchDB. Two differents technologies

I almost hated this database just because it was giving me some hard time. It was quite a journey. I am sharing with you today

  • couchbase: key-value database

  • Couchbase: "A distributed, memory-first, nosql, json document database"

  • Couchbase is not Couchdb Both open source couchbase offer more production features comparison table:

  • Couchbase vs Mongodb replace implementation using mongodb to couchbase

  • no eloquent driver for v3 for php sdk v3

  • https://docs.couchbase.com/php-sdk/current/hello-world/start-using-sdk.html

  • built new one based on eloquent driver v2 for php sdk v2 https://github.com/friendsofcat/laravel-couchbase

  • build container test runner on github action https://github.com/kirschbaum-development/laravel-test-runner-container/tree/master/7.4-couch-mongo

  • all coming together. CI checks all green, happy dev

  • I finally love couchbase ha

  • This is one common trait for us developer: liking a tech just because we're comfortable with it and disliking it because we lack clear understanding of the tool

  • Couchbase: "A distributed, memory-first, nosql, json document database"

  • using couchbase 6.x

  • migration guide 7.x

  • things I liked ACID, hearing multi tenancy

  • went from not liking it to loving it just because I understand it better and made open source contribution to that

  • looking for feedback on those contributions

  • difference between N1QL and SQL ?

    • goal being to know what N1QL is doing better than SQL if that comparison hold place
  • eventing service like firebase snapshot ?

    • this could remove the need of having pusher or application
  • use ephemeral database for testing

  • free training at couchbase

  • prepare documentation for other developer, as steps

  • better leverage couchbase cluster > bucket > scope > collection > document

    • cluster -> cluster
    • bucket -> database
    • scope -> schema
    • collection -> table
    • document -> table row
  • scope as a way to group with data with same structure (schema)

  • scope as a way to isolate data (like a view)

  • flexible cluster topology adjusts with growing demand

  • use one technology for our storage, analytics and caching layer

    • question: redis
    • question: elasticsearch
  • laravel eloquent driver

  • container for CI work on GitHub https://github.com/kirschbaum-development/laravel-test-runner-container

  • NoSQL option to store some json payloads from an external API. To use it as a "data-lake" to parse info from as needed

  • Metadata storage of JSON payload

  • analytics

  • challenges

    • setup/install
    • C SDK before PHP SDK
    • all in one setup script
      • cluster creation
      • bucket creation
      • primary index
    • docker container that does it all with proper version
    • tests
      • ephemeral bucket
    • uses laravel builtin command system
    • ephemeral
    • arbitrary decision
      • bucket, scope, collection vs database, table
  • wrap up

    • couchbase have more features than what is exposed with this work. The eloquent driver limits itself to just
    • a lot of arbitrary decisions taken
    • TODO
      • read replica to speed up
  • references

CI setup

#!/bin/bash

# Should match vars in .env.ci (ie COUCHBASE_DB_*)
username=Administrator
password=password
host=couchbase
port=8091
bucket=canvas
version=7.0.*

# TODO: move step 0 to laravel-test-runner container
## Step 0 - Because we don't know how to access binary from the couchbase container in github action service
echo "Install couchbase required tools for cli usage below"
curl -O https://packages.couchbase.com/releases/couchbase-release/couchbase-release-1.0-amd64.deb
dpkg -i ./couchbase-release-1.0-amd64.deb
apt-get update
apt-get install -y couchbase-server=$version
export PATH=$PATH:/opt/couchbase/bin

echo "couchbase-cli version: $(couchbase-cli --version)"

# Create cluster
echo "Creating couchbase cluster"
couchbase-cli cluster-init -c "$host:$port" \
          --cluster-username $username \
          --cluster-password $password \
          --services data,index,query,analytics \
          --cluster-ramsize 2048 \
          --cluster-index-ramsize 256

# Add new bucket
# Can also be done using BucketCreate command from kirschbaum-development/laravel-couchbase

echo "Creating couchbase bucket $bucket"
couchbase-cli bucket-create -c "$host:$port" \
        --username $username \
        --password $password \
        --bucket $bucket \
        --bucket-type couchbase \
        --bucket-ramsize 1024 \
        --enable-flush 1

echo "CBQ version: $(cbq --version)"

# Create index to allow n1ql queries
echo "Creating couchbase primary index on $bucket"
cbq -u $username -p $password -e "http://$host:$port" -s "CREATE PRIMARY INDEX "${bucket}_index" ON $bucket"

echo "Checking indexes state"
cbq -u $username -p $password -e "http://$host:$port" -s "SELECT * FROM system:indexes"

The command above are more suited to run as migration

Conclusion

  • thought at first it was too much for me to try
  • after working on couchbase driver turns out not that complex
  • just need cool head and clear thinking
  • next up: neo4j driver