Samourai Dojo can install Fulcrum if you set it to.

When installing fulcrum it just calls the Fulcrum install script.

The Fulcrum install script calls to install rocksdb and jemalloc (!).

The version of jemalloc that gets installed is the default, missing important flags during the build.

I believe this is because it is not even building jemalloc, but grabbing a precompiled binary.

https://github.com/jemalloc/jemalloc/issues/2624

--with-hugepage=21

https://github.com/jemalloc/jemalloc/issues/2639

--with-lg-page=16


All of this happens in docker files.

You could try to manually build/install jemalloc with the flags you want, so that when fulcrum is called to install jemalloc if it detects that it is already installed it may skip installing it again. 


dockerfile

# Use an official Debian Trixie base image
FROM debian:trixie

# Install dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    wget \
    autoconf \
    libtool \
    && rm -rf /var/lib/apt/lists/*

# Set environment variables
ENV JEMALLOC_VERSION=5.2.1

# Download and extract jemalloc
RUN wget https://github.com/jemalloc/jemalloc/releases/download/${JEMALLOC_VERSION}/jemalloc-${JEMALLOC_VERSION}.tar.bz2 && \
    tar -xjf jemalloc-${JEMALLOC_VERSION}.tar.bz2 && \
    rm jemalloc-${JEMALLOC_VERSION}.tar.bz2

# Configure, build, and install jemalloc with the specified flags
WORKDIR jemalloc-${JEMALLOC_VERSION}
RUN ./autogen.sh \
    && ./configure --with-hugepage=21 --with-lg-page=16 \
    && make \
    && make install

# Clean up build dependencies and source code
WORKDIR /
RUN rm -rf jemalloc-${JEMALLOC_VERSION} \
    && apt-get remove --purge -y \
    build-essential \
    wget \
    autoconf \
    libtool \
    && apt-get autoremove -y \
    && apt-get clean

# Set library path
ENV LD_LIBRARY_PATH=/usr/local/lib

# Default command to show installed jemalloc version
CMD ["jemalloc-config", "--version"]

----------------------------

This Dockerfile will create an image with jemalloc installed and configured with the specified flags. You can build the Docker image using the following command:


docker build -t jemalloc-custom .

And run it to verify the installation:


docker run --rm jemalloc-custom








---------------------------------
Also, weird how you can't get temperature unless you use sudo to run vcgencmd




-----
How to clean up a docker system

------
One thing that left me thinking was that fulcrum takes a long time to initially load, and it uses more memory than other indexers options in dojo. The main advantage is that it is very fast, especially with deep wallets (meaning hundreds or thousands of transactions)... Fulcrum was designed and written for BCH Bitcoin Cash where people were spending money like cash, and a wallet could have many transactions. But BTC Bitcoin is more like a savings vault, and there isn't a lot of transactions. So what is the reason to use Fulcrum? 
Also, if there is a sudden power failure, it can corrupt the WHOLE fulcrum database. 








What follows is just research and sample dockerfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# Use debian trixie for gcc13
FROM debian:trixie as builder
 
# Set work directory
WORKDIR /download
 
# Configure build container and build llamafile
RUN mkdir out && \
    apt-get update && \
    apt-get install -y curl git gcc make && \
    git clone https://github.com/Mozilla-Ocho/llamafile.git  && \
    curl -L -o ./unzip https://cosmo.zip/pub/cosmos/bin/unzip && \
    chmod 755 unzip && mv unzip /usr/local/bin && \
    cd llamafile && make -j8 LLAMA_DISABLE_LOGS=1 && \
    make install PREFIX=/download/out
 
# Create container
FROM debian:stable as out
 
# Create a non-root user
RUN addgroup --gid 1000 user && \
    adduser --uid 1000 --gid 1000 --disabled-password --gecos "" user
 
# Switch to user
USER user
 
# Set working directory
WORKDIR /usr/local
 
# Copy llamafile and man pages
COPY --from=builder /download/out/bin ./bin
COPY --from=builder /download/out/share ./share/man
 
# Expose 8080 port.
EXPOSE 8080
 
# Set entrypoint.
ENTRYPOINT ["/bin/sh", "/usr/local/bin/llamafile"]
 
# Set default command.
CMD ["--server", "--host", "0.0.0.0", "-m", "/model"]


Comments

Post a Comment

Popular posts from this blog

 The new RaspberryPi 5 has arrived and I've stuck it in a new Argon40 V3 M.2 NVME case. The reason for this was that after running Umbrel for 3 years the 1 TB NVME drive was up to 800GB and it was estimated there were about 10 months left before it filled up completely. This was mostly due to Taproot (which combined with Segwit) allowed for an explosion of blockchain bloat since last year. Umbrel had just released version 1.0 of their software and I was trying to figure out a path forward. Should I just upgrade to version 1 and buy a bigger SSD and continue running the rPi4 8GB? Should I upgrade all the hardware? Should I switch the OS to another platform? I felt that 1) having a bitcoin node to use was essential to maintaining sovereignty and 2) Umbrel was very successful, they had taken an open source project and commercialized it. I had watched it grow up from being a bitcoin node to adding apps and becoming a home server project. Most recently they were selling their own hardwa...
Read more
 Setting up the system. The first place I had turned was  https://wiki.ronindojo.io/en/setup/V2_0_0-upgrade-raspberry This website was soon also taken down as it used an install script that pointed to the Samourai.io servers (which no longer worked). I started by installing RaspberryOS which is based on Debian Stable (bookworm).  Then it was installing dojo based on the GitHub of  https://github.com/Dojo-Open-Source-Project/samourai-dojo/tree/master Then I found out that dojo is basically just scripts that install other programs. One of these programs is fulcrum - an indexer. fulcrum requires installing jemalloc. It didn't build jemalloc but just grabbed pre-compiled binaries. jemalloc would not run since it uses 4kb page size, and the rPi5 was optimized over the rPi4 to use 16kb page sizes.  So I had to change the boot kernel from the default to a generic kernel8 which uses the 4kb page size. Then I had a database corruption for fulcrum due to a power outage. I...
Read more