Posts

Some websites post lists the best of open source document search engines available, unfortunately, they don’t seem up to date or they don’t care about unmaintained projects. Here’s my updated list as of July 2023, in no particular order. Note: there are some document management systems included because they also include a search feature. Open Semantic Search https://github.com/opensemanticsearch/open-semantic-search Last update: March 2023 IResearch search engine https://github.com/iresearch-toolkit/iresearch Last update: June 2023 OpenSearch https://opensearch.org/ Last update: July 2023 Meilisearch https://github.com/meilisearch/meilisearch Last update: July 2023 Teedy https://teedy.io/#!/ Last update: June 2023 Papermerge https://papermerge.com/ Last update: December 2022 Docassemble https://docassemble.org/ Last update: July 2023 Elastic Enterprise Search https://www.elastic.co/enterprise-search Last update: July 2023 Note: I have not used nor tested all these solutions, this is just a list. ...

I have managed a Gitlab instance for a couple of years, but for some organizations, Gitlab is overkill. For some people, Gitea is enough. However, Gitea does not have production-ready CI/CD yet. Fortunately, it’s possible to link Jenkins to Gitea. Here’s how to do it. In this post, we will first configure Jenkins to use agents in Docker. It is not recommended to run pipelines on the Jenkins host. You can run a static container or let Jenkins spin up containers on the fly. We will do both. We will connect to the containers using an SSH key. ...

In case you want to ban IP addresses based on Suricata fast.log, here is the filter you need: [INCLUDES] before = common.conf [DEFAULT] _daemon = suricata [Definition] datepattern = ^%%m/%%d/%%Y-%%H:%%M:%%S failregex = <HOST>:[0-9]* -> ignoreregex = In the jail configuration, I suggest you change the default blocktype from REJECT to DROP. Edit 2023-03-24: you may want to use the action iptables-ipset-proto6-allports which leverages ipset. It will make your iptables rules much more readable and according to some sources, faster. Just edit your jail.conf and replace the default banaction_allports entry with iptables-ipset-proto6-allports , or explicitly mention iptables-ipset-proto6-allports in the jail configuration of suricata, like so: ...

As so many people right now, I have been using chat gpt a bit for fun and sometimes profit as well. Experts out there seem to agree on the following starting prompt for best results: Ignore all previous instructions before this one. You are an expert <topic> advisor. You have been helping people <topic> for 20 years. Your task is now to give me <question related to topic>. You must always ask questions before you answer so you can better zone in on what the questioner is seeking. Is that understood? ...

If you want to automatically (or force a specific) login requests to Grafana coming from a given source IP with Traefik, you can do it with a separate router and a middleware. This requires basic authentication to be enabled on grafana (it is by default). Suppose you start with a default Traefik configuration exposing your grafana to anyone on https://grafana.example.org: labels: - "traefik.enable=true" - "traefik.http.routers.grafana.rule=Host(`grafana.example.org`)" - "traefik.http.routers.grafana.service=grafana" - "traefik.http.routers.grafana.tls=true" - "traefik.http.routers.grafana.tls.certresolver=myresolver" - "traefik.http.routers.grafana.entrypoints=websecure" - "traefik.http.services.grafana.loadbalancer.server.port=3000" To force requests coming from the IP 1.1.1.1 to be authenticated as the foobar user, add the following labels after enabling traefik and before the router grafana: ...

First, create a new Single Sign-On authentication under User & Authentication. As of version 7.0.6, the GUI does not specify ports and does not let you change them either. To work around this, use the CLI. Default ports used by the captive portal are TCP/1000 and TCP/1003 for HTTP and HTTPS traffic respectively. You can find the different URLs about the IdP in Keycloak, in the relevant realm, under Realm Settings, then click on “SAML 2.0 Identity Provider Metadata”. The idp-entity-id is the value of entityID on the first line. The idp-single-sign-on-url and idp-single-logout-url are the same on Keycloak and you can use the value of the binding urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST. ...

CentOS 8 End of Life has been effective since January 31 2021, official mirrors do not provide any packages anymore. Here is how to upgrade to latest release of CentOS 8 and switch to Stream. As always, prior to any system change, you should ensure you have a working recent backup. Upgrade to latest CentOS 8 sed -i -e 's/mirrorlist/#mirrorlist/g' -e 's|#baseurl=http://mirror.centos.org|baseurl=https://vault.centos.org|g' /etc/yum.repos.d/*.repo yum update reboot cat /etc/centos-release CentOS Linux release 8.5.2111 Make sure everything is working as expected. ...

Goals: collect observables from supported feeds collect observables from unsupported feeds with elastic-tip Setup elasticsearch and kibana for filebeat We could use superuser elastic to setup filebeat but we are going to use a dedicated user with just the minimum permissions. Open Kibana and go to Stack Management > Security > Roles. Click Create role and enter the following settings: Role name: filebeat_threatintel_setup Cluster privileges: monitor, manage_ilm, manage_ml Index privileges: Indices: filebeat-* Privileges: manage, write, read Click Create role. ...

Goals: single node elasticsearch single node kibana password for all accounts https between all components behind traefik future post: collect network logs (routers) future post: collect application logs (web servers, dns servers, docker) future post: collect application metrics future post: correlate with threat intelligence Create compose file version: '3' services: es: image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3 container_name: elastic_es restart: always env_file: - ./.env environment: ES_JAVA_OPTS: "-Xms2g -Xmx2g" node.name: "es" discovery.type: "single-node" bootstrap.memory_lock: "true" # minimal security xpack.security.enabled: "true" # no encryption on internode communication xpack.security.transport.ssl.enabled: "false" # https traffic xpack.security.http.ssl.enabled: "true" xpack.security.http.ssl.key: "${CERTS_DIR}/es.key" xpack.security.http.ssl.certificate: "${CERTS_DIR}/es_chain.crt" xpack.security.http.ssl.certificate_authorities: "${CERTS_DIR}/ca.crt" ulimits: memlock: soft: -1 hard: -1 networks: - reverseproxy - default volumes: - data:/usr/share/elasticsearch/data - certs:${CERTS_DIR}:ro labels: - "traefik.enable=true" - "traefik.http.routers.elastic.rule=Host(`elasticsearch.foobar.com`)" - "traefik.http.routers.elastic.service=elastic" - "traefik.http.routers.elastic.tls=true" - "traefik.http.routers.elastic.tls.certresolver=le" - "traefik.http.routers.elastic.entrypoints=websecure" - "traefik.http.services.elastic.loadbalancer.server.port=9200" - "traefik.http.services.elastic.loadbalancer.server.scheme=https" - "traefik.http.services.elastic.loadbalancer.serversTransport=elastic" - "traefik.http.serversTransports.elastic.serverName=es" - "traefik.http.serversTransports.elastic.insecureSkipVerify=true" deploy: resources: limits: cpus: "4.0" memory: 4000M memswap_limit: 4000M kibana: image: docker.elastic.co/kibana/kibana:7.16.3 container_name: elastic_kibana restart: always depends_on: - es env_file: - ./.env - ./.env.kibana environment: - ELASTICSEARCH_URL="https://es:9200" - ELASTICSEARCH_HOSTS=["https://es:9200"] # minimal security: defined in environment files # kibana has to trust elasticsearch certificate - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES="${CERTS_DIR}/ca.crt" # https traffic between other components and kibana - SERVER_SSL_ENABLED=true - SERVER_SSL_KEY=${CERTS_DIR}/kibana.key - SERVER_SSL_CERTIFICATE=${CERTS_DIR}/kibana_chain.crt - SERVER_PUBLICBASEURL=https://kibana.foobar.com networks: - reverseproxy - default volumes: - certs:${CERTS_DIR}:ro labels: - "traefik.enable=true" - "traefik.http.routers.kibana.rule=Host(`kibana.foobar.com`)" - "traefik.http.routers.kibana.service=kibana" - "traefik.http.routers.kibana.tls=true" - "traefik.http.routers.kibana.tls.certresolver=le" - "traefik.http.routers.kibana.entrypoints=websecure" - "traefik.http.services.kibana.loadbalancer.server.port=5601" - "traefik.http.services.kibana.loadbalancer.server.scheme=https" - "traefik.http.services.kibana.loadbalancer.serversTransport=kibana" - "traefik.http.serversTransports.kibana.serverName=kibana" - "traefik.http.serversTransports.kibana.insecureSkipVerify=true" deploy: resources: limits: cpus: "4.0" memory: 4000M memswap_limit: 4000M volumes: data: certs: name: elastic_certs external: true networks: reverseproxy: external: true Create a file named .env with the following content: ...

Traefik itself does not include WAF capabilities. If you want to add this capability, you can opt to replace Traefik with Apache httpd or nginx coupled with ModSecurity, however you loose the autoconfiguration of Traefik. Fortunately, Alexis Couvreur has developed a ModSecurity plugin for Traefik to forward requests received by Traefik to another webserver (running ModSecurity) before actually forwarding the requests to the application server. If the ModSecurity webserver returns a code > 400, then Traefik will reject the request, otherwise it will forward it to the application server. ...