Dev Log

How to kick CI: general note for devs - you don’t have to rebase to pick up ci fixes - if you just kick ci to run again (or do a dummy git commit –amend && git push -f) ci will run with a rebased copy for you.

Opened new PR!

Notes on main places where Event::PaymentFailed gets pushed:

• OutboundPayment::abandon_payment - gets called when the user calls ChannelManager::abandon_payment, but also used as a helper in retry logic (retry_payment_internal + handle_pay_route_err)

• OutboundPayment::fail_htlc - gets called whenever there’s an error after a payment is actually sent, i.e. either the recipient rejected the payment or some node along the route failed to forward

• OutboundPayment::check_retry_payments - main place payments are retried

• Finish places that need to be changed to take a reason

• Make draft PR

  • Questions: Are all these variants unnecessary?

    • Either could get rid of them or lump retry stuff under an enum? or just lump together?

  • Note: RecipientRejected currently also accounts for when there’s just a path failure (like in the intercepted onion tests, it’s not necessarily the recipient rejected, but there was a path failure)

  • Doesn’t seem like it’s easily differentiable from where PaymentFailed events are pushed - different retry strategies being exhausted, recipientrejected or forwarding failed

• Fix tests

  • First fix the individual places, then the generalized functions

    • Prob good to add payment path failed → see pathfailed event

    • In do_automatic_retries, the retry strategy timeout is different from the normal route_params/payment_params timeout

    • Another thing making it hard to do this is for example do_automatic_retries AutoRetry::FailOnReload - how would we know if we reloaded?? (maybe this is answerable, but I’m defaulting here)

    • Uh in no_extra_retries_on_back_to_back_fail is the bs_fail_update actually mean bs? Like is this correct for the failure reason to be recipient rejected?

  • Two approaches: 1. try to refactor current test methods to include checking for a reason, 2. just write a separate test checking all the reasons and ignore it in current tests

  • No tests for RetryParameterError or RetryAmountOverflow (in outbound_payments.rs)

Cryptopals challenge 31 - timing attack.

Trying out this issue

• outboundpayment.abandon_payment is called in many different places

  • For this we’ll probably have to create another function - abandon_with_reason

  • In retry_payment_internal there are a bunch of fail conditions - is it useful to actually communicate all these? Maybe? Otherwise could just lump under a RetryError variant and tell them to check logs

• outboundpayment.fail_htlc is also called in different places

  • this is mainly called by fail_htlc_backwards_internal which is called in many places - hard to narrow down when source = Outbound

  • sigh, need to add another function here too?

  • or maybe can do smth fancy like return the event or something and push the thing with the reason outside this function?

• check_retry is really only called in one place ✅

• Need to fix tests - mainly the macro expect_payment_failed

• Important thing to note

  • outbound_payment::send_payment_internal is the only one that uses the handle_route_err which is the one that abandons based on PaymentSendFailure because it will want to retry - this is only used when you use send_payment_with_retry

    • For immediate errors this instead sends RetryableSendFailure

  • OutboundPayment::send_payment_with_route is the one that’s used in the normal ChannelManager::send_payment which will not do any retries for you and will immediately return the PaymentSendFailure on err

Review

• Main things changed:

  • Added optional_vec as a type for the TLV macros because vec_type always expected to read some value even if the vec was empty, and when you want to be backwards compatible and add a field that’s a vector, you can’t expect to read something (as it wouldn’t be there if reading from something persisted with a previous version).

  • Added nondust_htlc_outputs to ChannelMonitorUpdateStep::LastestHolderCommitmentTx and changed the main handler of that event provide_holder_commitment_tx to include some debug logic and also added some test stuff.

• This is a transitioning PR so no logic was really changed, but seeds were laid to make it easier to transition.

• General idea from what I understand is that we want to remove htlc_outputs field from LatestHolderCommitmentTx because it’s redundant because channel monitor update persistence holds us back from moving the channel forward in real time so we should focus on making it size efficient.

Learned about dust HTLCs.

• I was wondering why the dust limit even exists, and came across this mailing list post. Dust UTXOs increase the size of the current UTXO set without actually being valuable enough for anyone to want to keep there (because they won’t be profitable enough for anyone to spend). The size of the UTXO set is the main scalability concern, and the lower the dust limit, the more this is threatened (and for mostly negligible upside/flexibility added).

• Dust HTLCs are just HTLCs with value below the dust limit, so they can’t be enforced on-chain, and therefore have no corresponding outputs in commitment transactions. Like any msat denominations, they operate in a semi-trusted way between nodes.

• This led to me learning about OP_RETURN, which allows people to store arbitrary data in unspendable UTXOs that can be pruned. It was a compromise for people trying to store arbitrary data in other parts of transactions. Drawing this connection myself, I think I’ve heard of people trying to use OP_RETURN to encode more consensus rules (arbitrary data allows for more things to be validated or not, although since it must be unspendable I imagine that limits things).

Why is utreexo not more popular? Seems like it could basically enable full nodes on a phone (I might imagine bandwidth and processing power(?) might still be constraining, but still)?

Review

• What are temporary_channel_ids?

  • Channel id - 32 bytes, first 30 bytes are funding txid, last 2 bytes are output index (all big endian). This is created upon a funding_created message. Interesting to note this does not include the block height like short_channel_id as it’s created before the funding tx has been confirmed.

  • temporary_channel_id is used to identify a channel before the funding tx is signed, i.e. upon an open_channel message.

  • On a Channel object in rust-lightning, upon creation/channel opn channel_id is said to a temporary_channel_id, i.e. a random 32 bytes, and then upon the funding tx being signed, it gets changed to the actual channel_id.

• What are user_channel_id and former_temporary_channel_id in Event::ChannelPending for?

  • user_channel_id: no special meaning and exists only to allow users to have a persistent identifier of a channel. Exists in other forms of channel data in rust-lightning. Makes it easier for the user to identify, makes sense to be included in this event.

  • former_temporary_channel_id: Why is this here?

• Was confused by this comment that talked about pushing the event after the outbound channel end broadcasting/completing the channel monitor update. Still a little confused on the wording of waiting for the “broadcast" because I don’t think there’s any hook that would notify ChannelManager about seeing the transaction in the mempool or getting a message from the counterparty saying they broadcasted.

  • Generally I would imagine this would mean putting this in handle_monitor_update_completion? Actually there’s a function ChannelManager::handle_channel_resumption which is more general (that is used inside of handle_monitor_update_completion, but also in channel_reestablish) that does several checks (including broadcasting the funding tx ourselves) once a channel is resumed or a channel monitor update completes, which makes more sense to push the ChannelPending event here.

• How do 0conf channels work in rust-lightning?

Review

• Summary

  • Currently as a part of the onion payload the user only exposed data that’s received is payment secret. We want to generalize this to also be able to include payment metadata, and eventually custom TLVs.

  • I remember coming across “contexts” in the repo and not knowing what they were. Just read BOLT 9 for the first time. Feature bits are communicated in certain messages where it makes sense as a part of the payload (mainly init messages, gossip announcements, and invoices - these are the different contexts for where certain feature bits are necessary). Bits are added to the protocol in pairs, documented in BOLT 9 (even bit means requires, odd means optional, similar to how TLV types work). This makes the protocol very flexible in allowing nodes to implement new features and signal to others their support without anything special.

  • Looked at the [InvoiceBuilder](https://docs.rs/lightning-invoice/0.22.0/lightning_invoice/struct.InvoiceBuilder.html) for the first time and it’s so cool. They use type parameters to conditionally expose functions to the user based on the state of the invoice, so for example, it won’t compile if you try to call .build() on an InvoiceBuilder that hasn’t had certain fields set.

    • PhantomData can be used inside of structs if you want to use lifetimes or type parameters for compile time checks without any of the struct’s fields actually using the lifetime/type parameter (as by default rust doesn’t allow unused lifetimes/type params).

Got this error for this rust macro:

macro_rules! extend_secret_prefix_mac {
    ($mac:ident, $message:ident, $extension:ident, $digest_to_state:ident, $pad:ident, $hash_from_state:ident, $check:ident, $default:ident) => {{
        let mut new_mac = $default;
        let state = $digest_to_state(&$mac);
        for key_len in 0..=64 {
            let full_input_len = key_len + $message.len() as u64;
            let padding = $pad(full_input_len);
            let digest = $hash_from_state($extension, full_input_len + padding.len() as u64, &state);
            let final_message = [$message, &padding[..], $extension].concat();
            if $check(&digest, &final_message) {
                new_mac = digest.clone();
                break;
            }
        }
        new_mac
    }};
}

It gave me this error that digest and final_message didn’t live long enough to be borrowed. I thought this was odd because the function I modeled this after didn’t face the same problem. I learned that here it doesn’t know if $check will use these functions after the loop iteration is done, which I’m still sort of confused why that’s the case but it does explain the error. I asked GPT if I could tell the macro somehow that it doesn’t use these values and it said there wasn’t a way other than changing the $check function to take ownership of the value instead of references. Hmph. Feel like I’m using the macro wrong, like I’m treating it too much as a function that I want to pass functions into without making a generic interface for those functions heh…But it feels like I should be able to somewhat easily generalize this length extension attack to different hashes the way I’m intending but I just don’t know how.

Would be cool if there was a cryptopals equivalent specific for Bitcoin. Could call it bitcoinpals.

Do watchtowers claim revoked HTLCs? (not like claiming using the preimage, but like if the HTLC tx confirmed would a watchtower take that? (I guess the watchtower should’ve just revoked before that?)

git clean -fdx - get rid of untracked files (force, directories, ignored files)

Cryptopals challenge 30. Same MAC length extension but with MD4. MD4 and SHA1 share a lot of similarities. They use the same bitwise operators while scrambling, merkle damgard construction (which is why simple secret prefix MACs based on this suffer from length extension attacks), and pad to multiple of 512 bits and operate on 64-byte chunks. Main differences: SHA1 used big endian for any encoding/decoding while MD4 uses little endian, and some variation in the scrambling.

What happens if you get a payment (you’re the final node), then when trying to resolve the HTLC, the immediate downstream node goes offline and now you want to fail it. Can you still communicate to the other nodes to fail the HTLC back so they aren’t stalled?

PTLCs are so cool! Did not actually understand how they works in a full multi-hop payment flow, just the general idea, but went back to review today.

• They use adaptor signatures to ensure that no downstream channel partner can claim the payment until their upstream partner has claimed (which is how it works in the normal case for HTLCs, but it’s possible for downstream nodes to claim if they have the preimage, but in PTLCs it requires an adaptor signature directly from their upstream node). This also allows for a sender to include an extra adaptor secret and send it in the onion for the sender so that for a malicious lnurl situation where an LSP reuses an invoice, each payment has some secret that is essentially unique to that payment (even if the invoice was the same) and the LSP couldn’t steal funds.

• I didn’t realize that the Point in PTLC doesn’t mean a public key, but the adaptor secret (which are “masked” as points basically), i.e. the adaptor secret is what locks and is revealed for claiming, not like ephemeral keys or something.

Reviewed the async payment proposal again and it makes a lot of sense now. My takeaways (basically rewriting the main points in my own words):

• Receiver gives their LSP pre-made invoices, LSP holds the payment for sender, sender sends message to receiver LSP, receiver comes back online and pings their LSP to ping sender LSP, payment goes through. To prevent malicious LSP reusing invoices, sender includes an extra adaptor secret and sends in onion so there’s a unique secret per payment.

• This results in no liquidity lock up except for the sender, and no one being able to steal funds.

Figured out cryptopals challenge 29. Sha1 mac length extension attack. As long as you can guess the length of the original message (which is a simple loop), given a secret-prefix SHA1 mac, you can extend the message. I think you could block this attack however if you parsed for an ending suffix (i.e. only read/authenticate the message up to this suffix and make sure not to expose whether this was the issue to the attacker as that differentiation may lead to some shenanigans (as we’ve seen in many other attacks)).

Possible learning side quests for getting better at rust

• Revisit the book, rust by example, and rustlings

• Learn haskell (I’ll just wait for class next semester…?)

• Read ultralearning

Struggled on cryptopals

Had a lot of meetings

More stuff on my PR

• Make sure amounts work for both MPP and non-mpp

  • wait but…what if you have a double claim situation, but you overpaid the first time, and now you send a different amount - if you set total_msat to the actual amount upon pushing PaymentClaimable, you’ll refuse to claim the resent payments

  • Why keep the check in claim_funds as < rather than ≠: because in case HTLCs timeout and the sender resends in between generating PaymentClaimable and the user responding to the event, you’d still want to

• Writing tests is still hard but I’m getting better at it!

Review: ref is used in pattern matching when you want a reference to the inside value as opposed to matching/dereferencing a reference with the & sign.

Spent a bunch of time time writing some tests that didn’t get used :)

Did a bunch of refactoring of my cryptopals repository. Separated challenge implementations into modules by topic while leaving the actual tests in main to have a single place with every challenge in order so you don’t have to go searching for them. Also now that I’ve done a bunch of similar challenges I can generalize some parts to make things cleaner.

https://wtfhappenedin1971.com/ - When Nixon paused gold redemption and the US dollar went completely untethered from gold for the first time

Turned off diagnostic messages in nvim.

Reviewed a comment on my PR.

What is a secure enclave that they’re talking (in the LDK sync) about for running VLS?

channel_reestablish is another type of message in case peers disconnect while sending messages.

Ok, finally a conclusion on when to rebase, you should rebase…only if you have conflicts, after a new release (if you haven’t rebased since then), or if you need a fix in upstream

• Force push is good for really smaller stuff, but you should default to fixups and let other people tell you to squash them, as when you force push multiple times (without fixups) you lose the ability to see

• If you just rebase whenever you can, it’ll make it difficult to view the changes as it’s hard to tell what you changed from the previous state of your branch when you forced pushed if it includes a bunch of stuff from the rebase

Did cryptopals challenge 26. This was the bit flipping attack but with AES CTR, i.e. given an oracle that prepends and appends text then encrypts (where you can’t include certain characters in your plaintext), create a ciphertext with a target text injected in the plaintext. As we’ve seen with CTR mode, if the attacker has access to an encryption oracle, they can easily inject any text they want as long as they know what position their text is it, as the ciphertext is just the plaintext XORed with the keystream, so the simplest way would be to input zeroes, then xor that section of the ciphertext with whatever text they want.

Was reading quite a bit about the history of the federal reserve. Oversimplification, but it seems that throughout the 19th century, banks were playing it too risky and there were frequent collapses.

• My question: it seems like there are actually very different alternatives to fractional reserve banking, and I don’t know why they never worked out.

• The problem: banks need to make money, and they noticed they usually had a lot of money laying around not being used, so they started using it, but that risks customer’s funds, and when they risked too much and failed, they lost customer’s money and people were mad about this, however there was no alternative to secure your money than a bank, so as customer you had no options.

• Ideas for solutions: hold 100% reserves and charge a fee for securing assets. Seems like it would literally solve everything…but banks probably wouldn’t make nearly as much money (haven’t done calculations), so they never wanted to. Could also regulate banks more, maybe require a higher reserve ratio. Insurance was a good idea. Most importantly, banks need to fail to disincentivize risking customer assets as much as they do; let free markets play out. I really got to acknowledge this is a tough problem however, because letting free markets play out means real suffering, but in the long-term it’s probably better. Also importantly, this makes me really appreciate the purpose that Bitcoin can serve in providing secure custody of funds with no counterparty risk.

• Instead what happened: people asked or it just happened, the government intervened by establishing the Federal Reserve which would have more central control over banking in order to provide liquidity when needed, and eventually control the supply of money. The initial function of federal reserve was to have federal banks that could lend to banks in different states in order to utilize the larger pool of capital to assist banks in times of need. This wasn’t inflating the supply of money to do so, but this was the first form of a bailout that incentivized (or primarily didn’t disincentivize) more risk/debt.

Continuing to understand the Age of Easy Money

Did cryptopals challenge 25. If the attacker has access to an AES CTR encryption oracle that reuses the same nonce and key (where it can provide the input and get a ciphertext), they can simply find the keystream by providing an input of all zeroes.

• I’ve come to the conclusion that for AES CBC you need to include the initialization vector with the ciphertext (and same with CTR and the nonce). The receiver needs this information to decrypt, and yet as long as you use a different IV or nonce with each message (which is very likely if you just choose randomly), even if an attacker has access to the ciphertext with IV or nonce, they will not be able to decrypt (because they don’t have the key). In cryptopals and in the wikipedia descriptions of implementations there not super explicity about this so I realized I haven’t really followed the protocols completely correctly.

How it happened

• 2008 financial crisis happened because for the most part (simplifying), banks were playing too risky and not doing their due diligence. When that risk faultered, everything crashed.

• Government bailed out the bigger banks as one way to help.

• Importantly the fed stepped in with quantitative easing, which was where they introduced new money in order to buy assets (government bonds, corporate bonds, mortgage backed securities, etc.) and lower interest rates in order to inject money into the markets to inflate things, i.e. encourage people to consume. Quantitative easing was radical and experimental. It seemed like it was working because prices were inflating, but in reality, the money wasn’t being lent to small businesses or making companies employ more people, it was just encouraging banks to purchase assets and ride the inflation wave.

• Once they stepped in with quantitative easing, it was hard to stop. They tried to “taper,” i.e. pull back on quantitative easing, but markets started to panic and throw a tantrum. The fed crumpled from the reaction. Went back to more easing. Meanwhile the QE was not actually getting any money to the middle and lower class.

• Eventually the fed pulled back a little bit, but only slowed QE, not actually reversing it.

• Jerome powell was put into office. He tried to do quantitative tightening, markets again through tantrum, so back to easing. Easy money became the norm, and it changed everyone’s time preference to be very high, i.e. they value the short-term much more.

• The result? The tech bubble: totally overvalued companies, companies emerging that burn tons of cash before turning a profit. Private equity firms start buying everything up, and because the short-term is so much more valuable, they’re incentivized to do things that are cheap to drive up selling prices in the short-term rather than more profitable in the long-term (and you’d imagine this would entail more actual value creation). This also led to buy backs, where instead of investing in their workforce or infrastructure or any actual productivity growth, companies would just buy back their stock to drive up the price and pump their shareholders’ bags.

  • Private equity was especially influential on housing. Build cheap that looks alright to min-max selling in the short-term. This excerpt struck me a while ago, making me think what’s wrong with capitalism? But it’s really more of a problem with easy money and high time preference.

    • Hmm, but how does hard money incentivize against this (does it even?)? When money is harder to come by (debt is less common), and money holds it’s value, debt is riskier, and you are incentivized to prioritize the long-term, i.e. you are incentivized to invest in actual sustainable growth as opposed to maximizing profits in the short-term at the cost of meaningful improvements to productivity. It’s still possible to do these short-term shenanigans, but it will be much less common when money is actually aligned with value.

    • The biggest thing about hard money is just that when money holds it’s value, you are less inclined to spend unless what you are purchasing is actually reflective of the value you’re getting from it.

    • This goes for lending as well, as lending is essentially purchasing an asset, that you are taking a risk on expecting it to appreciate.

• This turned into this dependency between the banks and the fed. The banks hold everyone’s money, and they invest large portions of it in the market (or lend it out). If markets are down too much, it means normal people’s wealth is endangered. Ever since the fed has turned on QE, they’ve essentially been largely responsible for the markets doing so well, so if they try to reverse it, the market throws a tantrum. This incentivizes the fed to never let the markets crash, and so inflation soars onward.

• So many different things I’ve come to see wrong with the world are really connecting from understanding these macro trends.

Fractional reserve banking is okay. But the risk that is being taken today, and government bailouts not allowing markets to correct themselves (encouraging more risk) is dangerous.

Some remaining questions:

• How risky did banks play before bretton woods? What regulations are there on how risky banks can play, and also the reserve ratio?

grok: to understand profoundly through intuition or empathy. Great word.

The whole thing:

• Stems from two things:

• Storing money in banks for security. Fractional reserve banking. Paper money. Government takeover.

• Before the dollar, there was gold. Most people stored their gold in banks for security. Gold was not easy to transport, so when people wanted to use gold as a medium of exchange, the banks issued notes.

  1. Banks have been our only way, and therefore the gatekeepers, to secure our wealth and access digital payments.
  2. Gold was not salable through space, so when people held their gold in banks, the banks issued notes for people to use as a medium of exchange. At a certain point, governments took control over the issuance of these notes, as well as the interest rates that banks could lend at (when why how did they do this?).

Watched The Age of Easy Money documentary from PBS

• Told the story of the fed from 2008 to now.

• The fed can’t be responsible for everything, and many things are out of their control

• The fed did a lot of things that ended up incentivizing banks and large shareholders to play riskier because over and over the fed would never hold strong and instead responded to the whines of the markets

• Initially the fed would inject money, and it would mainly end up going to the rich

• Incentives at some point got super misaligned, where money was being lended to businesses, but instead of investing in things to improve the productivity of the company, they would just “buy back” their stock and pump their shareholder’s bags because that’s all anyone cared about.

• There seems to be a lack of regulation on the risks that banks are able to take.

• Notes on this to be continued…

Generally understood today what fractional reserve banking is. Didn’t realize even checking accounts weren’t full backed by cash. Whenever you put any money in a bank, they are only required to hold a fraction of that money as reserves, i.e. at minimum the reserve ratio which could be say 10%, and take the rest of your money and lend it or invest it. The thing about this though, is whoever the money is lent out to then purchases something or invests it somehow and the receiver of the cash stores it in a bank again. This bank does the same thing, and so 1/reserve-ratio times your amount of money can actually be used and circulate in the economy at one time.

• I could sort of see how this could help people grow their businesses or productivity quicker, but it seems so risky. It seems like it creates a huge bubble, and once there’s any sort of downturn or failure and someone can’t pay back the debt in time, a whole chain of command loses money.

More to add to the long list of things to look into: Erlay and Minisketch

Review: anchor outputs are important for fee bumping commitment transactions, but why do you need to fee bump them? Because HTLCs’ cltv_expiry. Say you’re routing a payment, and upstream they claim the HTLC and you get the preimage, however your downstream channel partner goes offline and you can’t update to get a nice clean no-HTLC commitment tx. You have to broadcast the commitment tx on-chain

It’s weird, in my mind I’ve had this abstraction of HTLCs being this separate thing on top of commitment txs that when they resolve you form a new commitment tx, but HTLCs hold actual concrete outputs on the commitment tx. When you do update_add_htlc, you are signing a new commitment tx with HTLC outputs and revoking previous states. It’s like I understood the concept of how these things function, but wasn’t connecting them properly.

Feedback on my PR!

• Add test for comparing cltv_expiry values (currently tests passing on >, but should also test failing on <)

• Need to understand channelmanager.rs 3942

• Check cases in process_pending_htlc_forwards (particularly in the non-MPP case, make sure PaymentClaimable always shows the actual received amount), understand what’s happening, switch from incoming vs. outgoing values. Cover with tests.

• Note: so actually it reports the sender intended value instead of the actual value in both non-MPP and MPP case

What does it actually mean when an HTLC is failed back? I thought that it’s like the HTLCs get cancelled/resolved. Does that mean that they commitment_signed revoke_and_ack a new commitment without the HTLC? It must be right? No matter what, these nodes will have to watch for previous commitment transactions (particularly the added failed one), but at least they won’t have this HTLC pending anymore.

A watchtower/client and rust-lightning’s native chain::Watch watchtower differ is some ways. Chain/channel monitors maintain lots of state, so they can receive information in pieces. Also they have access to a lot more state -

A watchtower client built for LDK is really like a separate user add on

How does watching for chain activity actually work in rust-lightning?

Reread my Crypto Snapshot February 2022 article. I’ve really come a long way in my understanding of things. Interesting to see the beginning of me really questioning things in crypto.

SIGHASH_NOINPUT. Sighash flags can modify verification for what signatures need to commit to. SIGHASH_NOINPUT means a signature doesn’t have to sign the input of a transaction, and so you can change what output a transaction spends after signing. This is helpful for eltoo because it allows for channel update transactions to spend from any previous channel state transaction if broadcasted (or any transaction that uses this flag). However, on it’s own this would mean that previous channel update transactions could spend from future ones, so they introduce another restriction of sequence numbers (didn’t learn exactly how, smth with nLocktime).

• Question: is it possible that on-chain you have two outputs that spend to SIGHASH_NO_INPUT, and when you broadcast your transaction, someone could use it to spend from the other output? I guess the funds are still going to the same place but that seems dangerous?

Let’s say your channel partner publishes revoked state. Now you broadcast a penalty tx to punish them. Is it possible that they could broadcast a transaction with a high enough fee, that miner’s would prefer to wait the delay and accept that transaction instead? I guess in that case you could still increase your own fee, but this is still pretty bad right?

• I think this might be called a bribery attack? Think I saw something about this in MAD-HTLC

HSM stands for Hardware Security Module, it’s a physical device for extra security

This looks worth: syntax highlighting pager for git, diffs, grep

Coming back to this

• Still looking into it, but I think the suggestion for hooking EcdsaChannelSigner:validate_counterparty_revocation, although it has access to the commitment index and the seed (and I believe the pubkey to send to) needed to form a penalty tx, I don’t think it’s possible or at least practical/easy to also provide the commitment tx data or txid in this way. Also using this would be a sort of weird workaround required to use a watchtower because you’d need a key manager/channel signer supplied from teos.

• Thinking it makes sense to push an event with the necessary information, and then teos could simply provide a handler for the event (at least for this primary usage of notifying a watchtower of channel updates).

  • At first I thought it might be necessary to be able to configure whether this event is generated to avoid bloat for users that don’t want watchtower capability, but I realize there are many other events that might be ignored by a user, and I imagine adding this event won’t be very resource intensive.

  • Thinking to look into how this might be done, maybe in Channel::revoke_and_ack…

• Note: commitment txs can have no pending HTLCs, or have HTLCs as outputs, in which case you’ll need the HTLC data to form those txs to get the txids for a watchtower

Can you have multiple pending HTLCs over the same channel at once? Yes. How exactly does that work? Each HTLC is just a new output on your commitment transaction.

I never realized how impactful credit card fees are on merchants, especially a small business like a restaurant. Importantly, their profit margins are already so small, 3% off the total price is probably more like 10-20% (maybe more) of their income.

What is the role of lending in a world where Bitcoin is the dominant currency? It seems like debt and lending can be a genuinely helpful tool to help people increase their productivity in a way they wouldn’t be able to otherwise afford. But the hypothesis I’m coming to is that it probably won’t be nearly as desirable with sound money. People would probably more often save than borrow when your money holds or increases in value, and debt does the same. It’d probably still have it’s place however. In this way you’d still have counterparty risk, but I’d imagine the scale of impact from mismanaging risk would not be nearly as large as it is today.

Awesome explainer game on the evolution of trust!

Thinking more these days how the goal of developing non-custodial, trust-minimizing—whatever you want to call it—solutions is not trying to get everyone to use things in an absolutely trustless way, but to make using it in a trust-minimizing way is simple and convenient enough to become the default. There are so many aspects of trust that seem unavoidable when it comes to supplying the simplicity of UX required to allow someone as technically inept as grandparents to use. Take the example of lightning state. Maintaining lightning state on your phone is not good enough because your phone may break or you may lose it. You must have some sort of backup somewhere. Many people’s only access to the internet is their mobile device, so their only option for backing up their data is in some server run by someone else. In the event that they lose their phone, they will technically be at the mercy of this provider, i.e. they are forced to trust this provider (maybe it’s backed up in multiple places, that helps but you get the idea). The provider has the ability to take advantage of them here, which is bad, but they won’t be able to lose or withhold people’s money at scale in the way that’s possible with today’s custodial solutions.

Birthgap documentary on declining birth rates. It’s a three part series and this is just the first part, but the only overarching trends the creator could really draw were that the general attitude of people in countries with longstanding declining birth rates (Japan, Germany, Italy) were that they felt it was too much of a financial burden or time commitment and they needed to work longer or establish their careers. The main trend in historical events he drew was that these countries were hit the hardest by major financial crises (1974 oil crisis), and the US and other countries have followed a similar trend as these countries since the 2008 meltdown.

Trying to understand why this comment (how EcdsaChannelSigner::validate_counterparty_revocation is all that’s needed to allow a watchtower to function for each channel update) makes sense

• Crucial reading: Keys section of BOLT 3

• Watchtower needs to know the commitment_txid to watch for, and it needs to be able to form a penalty tx to revoke. What are the bits and bolts that could let you create these?

  • Let’s exclude thinking about HTLCs for now because I’m not sure they are included for watchtowers for privacy reasons?

  • To get the commitment txid, you need to form the commitment tx. What is the structure of a commitment tx?

    • (as holder, not watchtower)

      • Input: funding tx out

      • Output 0: to remote revocation immediately, OR to local pubkey with (relative) to_self_delay

      • Output 1: to remote pubkey

  • To form a penalty tx, you need the local revocation secret.

• How is the per_commitment_secret generated? Randomly generate a 32-byte seed, then start from index 2^48-1 (281474976710655), then run the following algorithm:

  generate_from_seed(seed, idx):
  	per_commitment_secret = seed
  	for bit in idx (47th -> 0th bit):
  		if bit == 1:
  			flip bit in per_commitment_secret
  			per_commitment_secret = SHA256(per_commitment_secret)
  	return per_commitment_secret
  

  And for the next seed, decrement the index.

  It’s really cool how this works, because it means you can derive secrets. Define a prefix as the numbers leading up to the last 1-bit in a number. If you have an index prefix||trailing zeros, you can derive any secret for indexes with the same prefix. This essentially means you only have to store at most n/2 secrets for n commitments.

• EcdsaChannelSigner::validate_counterparty_revocation just updates the index of the counterparty_last_revoked_commitment (or does nothing in the case of the InMemorySigner which I’m sort of confused by.

• Reviewing once again: CSV vs CLTV is relative versus absolute time. An important distinction: relative timelock is only helpful for spending unconfirmed outputs. A relative timelock on an already confirmed transaction is essentially the same as an absolute timelock (except maybe in a big reorg). Also important: OP_CSV enforces an nSequence of the transaction that spends this output, while nSequence is what actually dictates when the transaction can be mined. Same distinction for nLocktime vs OP_CLTV.

I would probably say I believe the adoption of stratum v2 and the development of lightning are the two most important things to be working on in Bitcoin at the moment. Many more important things but these stand out to me.

Fee sniping is basically MEV, and MEV is an incentive for centralization, as it provides an economic reward for cooperating to reorg. Back in my web3 days I just thought of MEV as having an advantage in trading because you knew what was in the mempool lol, but that’s really not it. It’s that miners are able to extract value by taking advantage of their control over the order of transactions being mined (and that’s especially enhanced when you have something like on-chain trading).

Trusted third parties are security holes.

No Boilerplate on mediation and yoga. He described it as being especially helpful for programmers because meditation helps you have more control over your thoughts (which can be distractions you can’t do away with by putting on headphones or deleting an app). He also described yoga as exercises that help you be able to sit for longer periods of time without discomfort. I have been struggling with both of these lately, maybe I should revisit.

Realizing in my example of YouTube manipulating my actions: would it really be helpful to see the source code behind youtube’s reccomendation algorithm? It’s definitely helpful to be able to control it yourself, but idk. I guess the big thing is it’s more transparent, and people being able to have videos surfaced how they choose is the important part.

https://github.com/anuraghazra/github-readme-stats

Never knew the ::<Type> syntax was called turbofish.

Review

• Main idea: If we have any public channels, we don’t need route hints. However if we recently opened a public channel, it won’t be announced for 6 confirmations (and propagated for probably 7), during which we would not provide route hints, even though the sender doesn’t know about this channel.

• What is a route hint? BOLT 11 invoices include tagged fields for data related to the payment, one of which is r, for route hints. A route hint is just information about a channel for nodes that are only accessible through private channels.

• Channel announcement requires funding tx to have 6 confirmations? Yep.

• We refuse to include route-hints if we have any public channels? Yep.

• Never knew the precise fields relating to fees, but for a channel update (complements announcement, provides information about relaying HTLCs using this channel for a particular node) there’s fee_base_msat and fee_proportional_millionths

  Formula for fee: fee_base_msat + ( amount_to_forward * fee_proportional_millionths / 1000000 )

• Did not realize how many edge cases there are for route hints

Wow finally got fzf integrated in my shell. Amazing

Forgot this, but pathfinding is performed (by the sender) from the recipient back to the sender. Why? Because if we are trying to pay someone, we typically want them to receive that exact amount, and we will pay extra for fees on top of that. If we want to do something like send our remaining wallet amount somewhere, i.e. find a route given a sending amount, routing from the receiver to the sender makes things a bit more difficult.

• Was thinking why don’t we just route from the source in cases like this? Discovered minimum-cost flow problem which is pretty much exactly this. There are apparently ways to solve this efficiently but I realized it’s probably more work to maintain two routing implementations than it is to approximate with a couple passes from destination to source (routing value - predicted fee).

What are short_chan_id aliases and why do I always see comments about them?

Beginning review on this PR

• This issue is mainly concerned with ChannelDetails::outbound_capacity_msat being inaccurate, which just straight up leads to failed payments, but is also important for any further routing magic, including finding a route given a sending amount which I mentioned above.

• There are several checks that need to be added, but this PR tackles one of them.

What prevents channels from misreporting capacity and inflating the money supply? Must point to block number txid output index

References vs. pointers in Rust, Rust docs on pointer primitive. I’d seen *const before but never really looked into it. This is a raw pointer in rust, and if you want to dereference it, you have to use an unsafe block, as the compiler cannot know for certain whether it will be null or not. Null actually exists in Rust for these raw pointers! It seems raw pointers are only really used for FFIs and/or working with libc, or maybe some very specific memory management I can’t think of an example for.

A question I’ve always had: why can’t you have variable sized arrays on the stack? Apparently it’s just a safeguard built into programming languages to avoid stack overflow. The stack is much smaller than the heap, and variable sized arrays risk using too much of the stack. I’m sort of surprised this is a thing. C will let you do very dangerous things, but it won’t let you allocate a variable sized array on the stack. I guess you could hack it with pointers and put data in contiguous memory addresses, but dang. I can even imagine that there could be many programs that might have a small variable sized arrays that would definitely fit on the stack, but those aren’t possible (elegantly) I guess.

Also apparently stack size is usually just set to a megabyte, and determining how much stack space is required for a program is very hard in the general case. Some people will try and fill the stack with dummy values and run the program and see if it overflows. Apparently programs usually overshoot the stack size, but it’s not too harmful to do that. Except in embedded systems it gets complicated. Programmers can set the stack size too, and apparently the concept of the stack and heap growing towards each other is a myth. The heap is not just much bigger than the stack, it can dynamically grow in size, meaning a program’s total memory usage is not statically allocated. I remember learning a lot of this in our systems programming class, but need to refresh specifics.

Wish I could take CS 241 again, that class had so much good stuff in it. Coursebook - pretty funny, the introduction includes a couple paragraphs on editors and the TA who wrote a section that includes he uses spacemacs lol. Unfortunately the course site seems to be down right now…(or I think I need to be on the school VPN)

Note: they uploaded example proposals from last year for Summer of Bitcoin

Thinking about inflation again. We might only perceive inflation in terms of how much our money is worth relative to the rise in prices of goods we typically buy, but this doesn’t account for the improvements in production. Over long periods, we don’t realize this inflation is on top of however much cheaper these goods have become to produce. Say we see a 10% increase in prices, but this good actually became cheaper to produce which would’ve decreased the initial price by 20%, so our money has actually inflated 37.5% when we only think it’s inflated 10%!

Mainly was working on adding a test to this PR today. Modified an update_add_htlc message by reconstructing an onion to modify its values to overshoot amt_to_forward to check that we use the onion value for MPP set calculation.

What is OP_RETURN? I know it was previously left to include extra data about the timestamp or something but people keep talking about it in relation to taproot?

Watched Imitation Game. Alan Turing really had quite a crazy life. Crazy foundational work in computing, I didn’t even realize breaking enigma was a government secret for over 50 years, he was put on hormone therapy for being gay, and his best friend died of tuberculosis when he was in primary school. I imagine there’s so much more but wow.

Haven’t fully understood what people mean when they talk about fungibility. I have seen the issues with fungibility when it comes to things like the dust limit (and don’t really know what to say about it besides it’s a problem and it’s sort of solved by lightning, but generally I don’t know if it can be solved or is considered something to solve), but more recently I realized what people meant in terms of privacy. If the identity of a public key associated with an output is known, it “stains” all future spends of that output because now they are tied to that identity. This hurts fungibility because some people might not want to be paid with a stained output, when all outputs are still real bitcoin. I’ve heard how coinjoin helps with this which makes sense, not super sure if it totally fixes it after several mixed spends?

Checked out some new candidates for summer of bitcoin from the LDK roadmap, however for various reasons, it seems they aren’t fit.

• Learned about dynamic commitments (update). Really cool. Basically want to upgrade format of commitment transactions without creating a new channel and funding transaction on-chain (also can update funding transaction by deferring it to closing). This has been needed for static_remote_key (apparently when force closing, your counterparty required a secret from you to claim their funds, surprising), as well as for anchor outputs, and now for taproot.

• Project scoping is difficult in general for a fixed timeline. Okay that’s really not necessarily true, but for some reason it is for me lol.

Dioxus, Rust UI that runs anywhere? I sound like clickbait

Waa aaahh

Stuff on elliptic curves - haven’t really looked into it yet, but it seems that multiplication/division with an elliptic curve generator point corresponds to exponents/logarithm with a modulo group generator.

• Fun visualization on desmos for elliptic curve addition. Elliptic curve addition: the sum of two points is where their slope intersects the curve, then reflected over the x-axis (negate y).

• Okay, I’m learning math. A set is a collection of elements, say, integers.

• A group is a way to describe structure (of a set), i.e. a group is a set and an operation, such that they have the following properties

  1. the operation performed on two elements of the set results in a third element of the set
  2. there exists an identity element
  3. every element has an inverse which means the operation performed on an element and it’s inverse results in the identity element
  4. the operation is associative (e.g. (A+B)+C = A+(B+C))
  • The most basic group is the set of integers and addition.
  • An abelian group is one where the operation is commutative (points on elliptic curves and their “addition” is an abelian group).
  • Bonus: a space can be characterized by their fundamental group, which is the set of “loops” (like a path that starts and stops in the same point I think) from a single point.

• A field is a group with a multiplicative identity and inverses for each element. Importantly a field’s multiplication operation should distribute over it’s addition, i.e. A*(B+C)=AB+BC.

  • Technically what we call addition and multiplication here don’t always have to correspond to “actual” addition and multiplication, but generally have the same properties so they’re often called so.
  • The most basic field is the set of rational numbers with basic addition and multiplication.

• Was suggested to see this paper on how discrete log exists in elliptic curves.

If you check the security info in your browser on an HTTPS connection, it’ll show it uses something like “TLS_ECDHE_RSA_WITH_AES_GCM_128_SHA256.” What does this mean? The root of trust is that you trust federated certificate authorities to verify clients’ and servers’ public keys. These public keys are used initially to sign messages. Then they use Elliptic Curve Diffie Hellman Ephemeral to establish a shared secret key, which is used for symmetric encryption (more practical/faster than public/asymmetric), AES, in GCM, which is just counter mode (full name Galois/Counter Mode - Galois because of the associated message authentication system that uses Galois (finite) field multiplication smth?). Still kind of unsure where SHA256 is used, because authentication and data integrity seems to be already provided with RSA signatures and the encryption scheme. Maybe for the PRF somewhere?

Even homebrew has telemetry by default.

Saw someone say on nostr how they take more of a payment by receiving payments on a private lightning node only accessible through their own routing node, so they get the routing base fee in addition to the payment. Seems like this shouldn’t be a thing…but I can’t tell if it’s a problem.

If I have the technical capacity and money to be able to secure my setup and protect my privacy, why wouldn’t I?

PTLCs - HTLC payment parts can be identified as they use the same hash, while PTLCs can use signatures to different points (i.e. locked by points on the elliptic curve) to be unlocked. Also with schnorr signatures, elliptic curve points can be aggregated so that the extra lock doesn’t take up extra space if claimed on-chain and obfuscates that different spending conditions are even present.

• Why can’t HTLC MPPs just use different hashes? Doesn’t payment_secret solve this? Think about how payments actually work. For an invoice payment, the preimage is generated by the receiver, who is agnostic of how the sender will route the payment, so for the sender to use different hashes, they’d need to provide the different preimages to receiver. Maybe a keysend MPP could use different hashes, but then I guess how do you tell they are a part of the same payment?

• I’ve actually sort of wondered why keysend doesn’t seem to be that popular? Maybe because no proof of payment?

Adaptor signatures - schnorr signatures allow for special combinations to hide and reveal information to help coordinate multiparty signature schemes.

• An adaptor signature is like a normal schnorr signature but with an added hidden value (message m is public) (optech also hashes public key, which I’m not sure why it would be necessary).

  // normal
  s = r + H(R || m) * p -> (s, R)
  // adaptor - t is a hidden value
  s = r + t + H(R + T || m) * p -> (s - t, R, T)
  

• Say Alice sends this adaptor to Bob. Bob can verify the adaptor but cannot use s - t as a signature in anyway because he doesn’t know T and he can’t modify s or create a new s to change what is being hashed.

• However Bob can create his own adaptor that when sent to Alice (or anyone who knows t) can reveal his signature on the original hidden value.

  s_b = r_b + H(R_b + T || m) * p_b
  

• Alice can verify the adaptor, and produce Bob’s signature on t by just adding: s_b + t. Alice has retrieved a signature on t from Bob without Bob knowing the value of t!

• Then if Alice ever broadcasts this signature, Bob will be able to find t, and therefore also Alice’s original signature s.

• Don’t know many examples of how these can be used, will look more into it sometime.

Stuckless/cancellable payments - 2 rounds to accept a payment. Currently, all the data needed to claim a payment is sent along a route (or generated by the receiver), so if a payment gets stuck and you retry successfully, that stuck payment could become unstuck and you end up unintentionally overpaying. If you require the receiver to make another request to the sender (to obtain some secret) in order to claim the payment, you can prevent this. Downside is 2 rounds uses more bandwidth and is slower.

Scriptless scripts - because schnorr signatures allow for signature/key aggregation, you can lock UTXOs on the possession of secret keys, who’s public keys can be aggregated with other public keys to eliminate the need for certain scripts used today.

Chaincode research website is awesome!

Forgot about this website: bitcoin development philosophy.

This is also fire: A friendly introduction to bitcoin

Okay, need to stop surfing links and get to work lol.

Made changes to my PR.

Stumbled across Bitcoin Core’s productivity notes.

cargo tree shows dependency tree of your cargo project!

Saw Code to the Moon’s video about emacs org mode, rather than being interested in Doom Emacs, I discovered corne keyboards which seem super cool. Agh, how to balance between actually doing stuff and optimizing my workflow!?

Was thinking more about my PR and received a new comment - decided to go with the suggestion. While doing this I explored the other LN implementations for the first time. It was surprisingly not too difficult to find what I was looking for, besides being in different languages and massive repos.

Did some OCaml with a friend’s homework. Pretty interesting.

New LSP from TBD, c=! (c= is meant to be speed of light, get it, because lightning network infra)

Learned vim tabs exist. Was thinking I was going to use harpoon, but this will be much simpler, and it’s straight out of the box

Probably the most compelling piece I’ve read on privacy. Another piece on bitcoin privacy pre-history.

I should maybe consider applying for a TABConf 2023 grant. Would be super cool to go.

Learned what mutation testing is. Almost like testing your tests. Make a particular change to a function, and if you’re tests still pass, it means your tests fail to cover a particular case.

Started exploring more for summer of bitcoin stuff - looked through other LN project ideas, started browsing more LDK issues and roadmap. Got distracted by Gigi’s 21 lessons.

It’s already March!

For a while I’ve had the thought that another prime example use case for a blockchain is something like voting for political election. Something that’d be best for anyone to be able to verify. But I’m realizing this probably doesn’t even need a blockchain, i.e. you don’t really care about order, you just care about the count. You would still care about consensus, and I wonder how you would deal with conflicts…Also really have no idea how you’d deal with sybil attacks here and also privacy probably requires some zero-knowledge stuff.

Updated my website finally.

Taking a bit of a break today because it’s really nice out and I’ve been feeling some back/eye strain from the past couple days.

Random tools I ran into: slides, excalidraw, graph-easy, lf

Clarifying for myself on intended behavior of test_dup_htlc_second_rejected. This doesn’t test if a second HTLC overruns the amount it will fail but the first will be claimable, it tests if a payment is already been claimable and you get a duplicate that the first will remain claimable and the duplicate will be failed (same things but nuanced)

Have not been doing a good job documenting last couple days. That’s okay! Been heads down making this PR!

Interesting, learned about different payment reversals: article, article. Authorization reversal is reversing a payment after it’s been initiated but not settle through ACH and this is the easiest/cleanest reversal. A refund is exactly what it sounds like: the payment has been settled, now the merchant is abiding by some policy and has to send the funds back in a new transaction. They lose out on the sale, the cost of the product they lost, as well as the fees for both transactions. A chargeback is as bad as a refund but worse because there’s a dispute. Merchants also incur extra chargeback fees, need to fight fraudulent claims, and if they face too many chargebacks the credit card networks can flag them and revoke their ability to accept credit cards. Financial institutions using the card networks are tasked with monitoring chargeback thresholds to prevent fraudulent merchants.

Wow. bmoney

There is a reccommended character wrap of 100 chars!

Started on 2 things

• MPP overshooting

• Summer of bitcoin project scoping

A tool I would really want

• Feed codebase into AI → ask questions

• Example questions: I’m looking to do this, are there functions available that do this?

I should maybe try harpoon, because my monitor isn’t exorbantly large, and rust-lightning got some long lines that make split screen not that great, and it is true I’m am very often switching between just a couple files on the fly

I am honestly quite impressed with my previous self with the test I wrote in my first PR. It’s a bit of a struggle navigating how to do exactly what I want because every function automatically does a bunch of checks that I might not necessarily want.

Mainly working on this all evening.

Finished set 3 of cryptopals! Challenge 24 was kind of weird. It was mainly creating a stream cipher with the PRNG and then breaking just with normal brute force which seemed sort of unfulfilling. I looked it up to see if I was misunderstanding and other people seemed to have come to similar conclusions.

I always hear people talk about how things are wrong with our financial system. I’ve never really understood this side. I get how inflating the money supply is bad, and I get how trusted intermediaries in payments cause issues, and how secure self-custody is important, but I imagine the type of experiences the people at a place like Square have encountered dealing with the financial system are different in how they reveal it’s flaws. I want to learn more about this.

Also want to understand more about the idea of “the blockchain only being used to settle disputes”

Rip neovim slowing down on this 10K line file, but helix doesn’t 👀

• In the short-term, for specific problems, other things can probably accomplish things better. If people just want to hold value when their country’s currency is failing, they can use a stablecoin. At the end of the day, there needs to be demand for a trustless censorship-resistant money in order to pay miners for the security budget and for bitcoin to succeed. In the short-term and for many cases, absolute trustlessness may not be needed. But odds are that there will be times when its needed, and for those cases it seems to me it’d be necessary for the foundation to be trustless. Will these cases be abundant enough to pay for a sufficient security budget? I don’t know!

  • At the same time it has a lot of interesting economic things going for it. Fixed supply means if other moneys continue to inflate, Bitcoin will outcompete others as a money, especially as a store of value. Bitcoin’s relationship with energy while block subsidy is present

  • If your bottom layer isn’t trustless, then the only way to make trustless payments would be to on-ramp to a separate currency, which could be censored. If you don’t on-ramp, you’d need to hold some of the trustless currency at all times in case you needed censorship resistance, which only makes things more difficult.

• Generally: censorship resistance! But many other things. Separating the money and the state. Replacing a predatory financial system.

Pretty fun video going over a bunch of tooling built in rust. Mentioned sccache which for a second I thought could be the solution to slow test compilation in rust but I realize I think it’s actually for caching compilation of libraries that’s shared between code.

• Also this guy No Boilerplate runs Asahi linux 🤔…maybe I should actually try it out.

Went to a talk by Benedikt Bunz that was happening at my school. He talked about Bulletproofs. So what I understood, a SNARK is a zero knowledge proof with a very small proof, however they require a trusted setup. A STARK doesn’t require a trusted setup, but is apparently very big. Bulletproofs are small and don’t require a trusted setup! The main example of usage of these is the make it possible to have private transactions on blockchains while still knowing that people can only spend amounts they actually have (i.e. range proof → amount to spend is greater than zero, less than some huge number). It was pretty interesting. I couldn’t explain all the technical details, but he talked about using homomorphic property of commitments (being able to essentially add them together and do batch verification) to do rollups, smth smth you could compress large amounts of data with commitments while still being able to maintain inner product argument. Don’t know much details of all these but was cool.

• The talk was generally about solving problems with blockchains currently. He also talked about how to use verifiable delay functions (VDF, e.g. hash chains) to do leader section to replace the “wasteful” proof of work. I’ll note here that proof of work has another important function beyond leader selection, i.e. requiring a costly action outside the system in order to produce histories, but the construction was pretty interesting. Main idea I got was you are able to have a group of people choose a random number that none of them can predict without trusting each other. They do this by all choosing some random numbers, then taking the hash of all of them, and then running a VDF on that hash. Pretty cool to get the basic idea of leader selection in proof of stake type systems.

• Interesting to note that proof of work is not a VDF. In VDFs, you can actually verify this must have taken exactly this amount of time (this amount of hashes) to compute, whereas proof of work it takes an unpredictable amount of hashes.

• These days I’m thinking about how proof of stake could maybe work in practice, especially overtime overtime as a blockchain becomes more decentralized, even though the trust model is different. But I don’t know it’s kinda like why leave room for error. Though from my understanding proof of work is more sound, and the relationship to energy is in my opinion a huge plus.

AstroVim. Vim out of the box. Cool, but eh not for me.

Fuzzers are smart! I realize now there’s much more to fuzzing than just random inputs. In hindsight it’s pretty obvious it would take a really long time to find helpful inputs if you were going off of pure randomness. Instead fuzzers will have some preset inputs, and you can even seed or template inputs, and they will mutate this initial set based off of code coverage (if it has access to the source code) and how each input performs. Should look more into this one day.

ThePrimeagen on github copilot. Agree. Even more helpful, he recommends not using it as a junior developer. I feel like I could probably use it somewhat effectively, but this just further confirmed my gut feeling to just not use it until I’m much more prolific.

Realized that the fact that I would be able to backup my Phoenix wallet must mean that ACINQ must store some sort of (all of the) state for my lightning node. I realize that backing up your lightning node state is fundamental to self-custodial lightning—if you lose your phone you should not lose your funds. But also I guess you don’t have to back up everything, you could only backup the latest commitment transactions and force close if you lose your phone. Not a great option though. Hello VSS!

Notes from Citadel Dispatch ep 90 with Steve Lee:

• Stratum v2 is already in production on Braiins pool (3% of hashrate). Importantly though an independent implementation is being funded which will make adoption easier, and also generally good for the protocol to not have a single implementation by a for profit company.

• Pavlenex is apparently one of the best examples of a non-developer contributing in a major way to Bitcoin open-source projects, e.g. BTCPay and StratumV2!

• People are really excited about fedimint, and although it’s sort of custodial, it federates trust, and has its use case. It helps scale bitcoin and interoperates with lightning. Haven’t taken the time to really understand it, but smth smth federated trust, smth smth chaumian e-cash. Gotta look into it soon.

• WalletScrutiny. Project to verify reproducible builds for wallets. Totally agree with the idea. Apparently it’s been sort of controversial just because there are a million wallets and very few people working on the project, and it’s easy to get on wallet developers nerves just from the nature of the project. Lots of nuance that isn’t totally accounted for yet in the project.

• Compact block filters are great, the main reason people use esplora/electrum for block source is because of bandwidth on mobile.

• The ideal is where there’s a large market for LSPs, and there are many to many relationships between wallets and LSPs.

• There’s bitcoin park in Nashville..?

• Programming Bitcoin from Jimmy Song was a great 2-day workshop to give people a foundation. People are working on a Programming Lightning.

Notes from Stephen Livera ep 461 with Matt Corallo:

• Most people in economies where their money has failed actually use stablecoins much more often than Bitcoin. Community needs to do some soul searching on what Bitcoin provides people in excess of stablecoins. Stablecoins are centralized, but for 99% of use cases where people just need exposure to some more stable asset, they’re fine.

• Fee sniping and fee smoothing. I never heard of this until now, but fee sniping is when in the future when transaction fees make up the majority of miner’s earnings, miner’s may be incentivized to do much longer reorgs for a single high transaction fee. Fee smoothing would just mean that the fees from a block payout over the next several blocks to disincentivize this.

I always forget so I’m writing it. Little endian is backwards from normal reading, when storing left→right or top→bottom, the little end is stored first. Big endian is opposite.

I realized a reason people may be so adverse to hard forks is because for the minority of people that might not realize to switch, they can lose money. In a successful soft fork, people’s nodes will automatically abide with the chain with new consensus, even if they will still accept blocks from old consensus.

• If a very small portion of the network is fractured off, then it becomes much easier for people to do 51%/eclipse attacks and double spend. Importantly, if someone doesn’t realize they’re on the wrong chain, someone could pay them on the wrong chain, and when they switch to the right chain, they haven’t been paid.

• It’s interesting, initially I had been exposed to the difference between hard and soft forks from Vitalik on Lex Fridman’s podcast, and he described soft forks as being coercive like a soft fork changes consensus from underneath people, while hard forks force users to make a choice. But the way I’m seeing it now, people might not be aware they even had to make a choice and end up unintentionally making a choice that costs them money. You need to have a very high certainty that every full node users are exposed to will know to make a deliberate choice in order to hard fork without risking people’s funds, and in a decentralized network, that’s most often not the case.

• The weird thing now that I think of it is that even though consensus is different, as long as the same transaction formats are accepted, people could still send and receive transactions from each other even if you’re on different chains, as the transactions would still be broadcasted on the same network, and both could see that the payment went through.

Saw Coinbase launched their new L2. Things like this give me belief that there are people that are working towards the same overall mission of economic empowerment, but genuinely believe in non-Bitcoin methods being the way. That’s cool, and I’ll be happy to see any success that it has.

It’s crazy, I’ve basically switched to Helix and then NeoVim mainly because of YouTube videos. There are so many other ways that my life has been significantly altered by YouTube. On one hand, I am mostly in control of whether and how much I use YouTube, and I’ve been able to utilize it to great benefit. But at the same time, it’s weird to think of how easy it would be for them to influence a massive population of people at the same time. Like they know a group of people that have similar interests as me, and they could just push a couple more NeoVim videos to each of us, and all of a sudden they’ve influence probably thousands of people to switch editors. Thinking of this is rather unsettling. My life generally feels free, but when I look at my behaviors as results of a centralized power, it doesn’t feel very free. I’m understanding more intrinsically the reason for decentralized social media.

In code, planning is often best informed by implementing!

I’m conflicted on debt. It seems like a natural mechanic that would exist in an economy. At the same time the way it’s utilized today seems to lead to unnecessary cycles that risk real suffering. I’m not sure whether it’s because debt should be limited by some sort of regulation, or that there is some outsized factor distorting how abundant debt is.

As a reviewer I am realizing how much nicer it is when PRs are self-contained. And making things easier to review is really helpful.

Checked out how fuzzing actually works in rust-lightning. Pretty cool, it’s mostly autogenerated and they use several different fuzzing tools. Mainly just have to implement your actual test, add a few lines to some config files and run a script. Whenever reviewing new fuzz, I realize I mainly should just look at fuzz/src, and fuzz/src/bin is mostly autogenerated.

Learned what x-only pubkeys are. Pubkeys are denoted as their point on the elliptic curve. Because for every x coordinate there only exist 2 y coordinates on the elliptic curve, instead of denoting public keys as both the x and y coordinates, they use a tiebreaker byte 02 or 03 prepended to the x coordinate. X-only just gets rid of this tiebreaker and defaults to the positive point, and it doesn’t make it any less secure!

Changing visibility with conditional compilation in an elegant way is sorta hard. Noticed here the trait was put in a separate module for this sake. Anything is possible with macros however (although still not super elegant)! It’s not that big of a deal though.

Found what it was called for safe overpayment to increase routing success: Boomerang payments! Also stuckless payments is interesting and could accomplish a similar thing.

I’m in this weird spot where I am learning a lot from reviewing a PR, but I have very little to comment or ask. Most of my time spent reviewing is just answering my own questions, and by the end I don’t have much to say lol. Maybe I just need to find PRs where I can be more valuable. Not a big deal.

Cool well produced videos explaining Bitcoin concepts.

I feel like I’m in a weird spot where I’m probably not helping thaat much

• Nah. I’m basically still new the repo. Can still help out with public docs, but can also ask more conceptual questions about code. Go through PR looking through these things primarily. Don’t need to comment on random stuff, just things that actually stand out.

• Still would definitely benefit from building some other stuff in Rust to grow more opinions about how it’s used. I can get this from reviewing and making PRs, but I think I’ll get it a lot faster through a side project.

Open sourcing code is sort of like peer review in research. A very small portion of the people who use the product or information found from research are reading the code or the paper, but other engineers and researchers will review it and point out if there’s something wrong. Just because something is open source doesn’t mean it’s totally secure (and just because something isn’t open source doesn’t mean it’s insecure), but it’s necessary to ensuring publicly that things are right.

Why does everyone comment “needs rebase” on other people’s PRs? Shouldn’t it be understood to intermittently rebase as a PR author?

When reviewing PRs in local branch - use git diff --merge-base main (or better replace main with FETCH_HEAD after fetching main to avoid having to git pull) to see changes in case it needs rebase. Add --stat to end of git diff to just get the number of line changes to each file.

Git stuff is all 80 char limits.

* goes to next occurrence of the word currently under the cursor, # goes back!

Review

• When are these params deserialized/why would they be? They are included in events, and events are serialized because they are persisted in case of a crash. So in this case, RouteParameters are serialized, because they are included in Event::PaymentFailed as they’re needed in order to possibly retry a payment. All events are serializable, but this one is important to be persisted because in case of a crash, a node would still want to know to respond to this event either to retry or abandon a payment upon starting up again (and all the necessary information to do so).

• Generally I’m realizing a lot of the difficulty of a lightning implementation is that your node could crash at literally any point, and you 1. should not lose your funds because of it and 2. your node should be able to resume whatever it was doing previous to crashing (as a part of not losing funds, but also for general UX). For non-custodial lightning the responsibility of maintaining all this data is on you/your node, so it’s important that your node’s software can automate this and make it as seamless as possible.

• You can define generics in a trait such that you can implement the trait with a function that accepts a different type for a parameter, e.g.

  pub trait FooTrait<P> {
  	fn read<R: Read>(reader: &mut R, params: P) -> Result<..>;
  }
  
  // impl
  impl FooTrait for Bar {
  	fn read<R: Read>(reader: &mut R, number: u32) -> Result<..> {}
  }
  

• The tt Token Tree is basically a “match anything” fragment specifier.

• ? along with * and + are used similar to regex for matching in macros, 0|1, 0..inf, 1..inf respectively.

How to store PGP keys safely?

Didn’t know about keystone, looks very solid.

Simple bitcoin inheritance: send to script that’s spendable by your heir’s key after a certain time, otherwise spendable by you until that time. Extend the time as you continue to live.

Can probably change cryptopals idea to just be a CLI that runs locally

• Can just take input or commands from stdin and run tests locally

Signal vs telegram open source - signal is full open source, i.e. they open source their clients running on your phone and the code on their servers (technically can’t verify they are running the same code but seeing the code gives higher assurances, it also means you could run your own signal server which is sick (and I’ve run a proxy before and if it’s similar, it’s fairly easy to switch what server your client points to), telegram does NOT open source their server code which is SUS but they have client code and their encryption protocol open sourced (and includes clear instructions for verifiable builds which I gotta pay my respects for) and honestly i feel like if people have audited the protocol then it shouldn’t matter what the server is running if it complies with the protocol it should be secure…although they may be collecting lots of other metadata on messages like WhatsApp which could be problematic.

For existing branches such as branches fetched from other remote repositories, create a worktree through the git worktree add <path> <branch> command. The default git worktree add <path> will create a new branch/worktree.

You can resize your window in Vim with :resize <char-count> and resize horizontally with :vert at the beginning

Our World in Data on democracy. Cool.

Love ThePrimeagen. “Discipline is freedom.” “This stuff is hard af but you can do it and be a badass.”

Dang I forgot february is only 28 days, march is so soon!

Realized soft fork activations risk network splits and double spends temporarily. When there is a fairly even split between those rejecting blocks based on new rules and those who haven’t soft forked, competing chains will last longer and people will be more easily subjected to double spends. When the vast majority of the network is switched over, then even if old nodes are accepting no longer valid blocks, they will still find the right chain as majority of the hashpower will be growing a single chain.

• The users ultimately decide what is valid bitcoin, but miners are needed to grow the work of a single chain, which is particularly important during a soft fork activation.

Taproot is cool! It introduced a standard for schnorr signatures on secp256k1 which has several benefits over ECDSA and also unlocks new abilities! It allows for outputs to commit to script hashes with multiple branching paths, but spend from any path by only providing the necessary data for that path, obfuscating not only whether an output was to a script at all, but also the construction obfuscates whether a payment is to a multisig or a single public key.

• The construction is as follows (as summarized on the mailing list proposal): C = A + B where C is the aggregated public key of public keys A and B, P = R + H(C || S)G where G is a base point, R is a random point (rG), S is the script, and P is the point that is paid to, appended to a version signaling Taproot.

• Two parties can sign (apparently all you have to do is add H(C || S) to one secret key to be able to sign for P = A + B + H(C || S)G??), or one person can sign for their path as long as S is provided and the conditions of S are met.

• Learned segwit v0 only had people sign for spending an input, but segwit v1 enforced signing an input plus the outputs.

• The full taproot upgrade is split amongst three BIPs, 340, 341 (segwit v1), and 342.

• There are many things in the future that will come from taproot. Cross input signature aggregation could aggregate many signatures into one, making coinjoin spending cheaper than a normal transaction. More advanced signing protocols - musig for multi-signatures (n-of-n), and frost for threshold signatures (t-of-n). Batch validation (validate many signatures at once, doesn’t matter which signature is invalid, just want to know if all are valid or not, perfect for Bitcoin).

• Definitely a lot more to be said but these are my notes for now.

• It’s interesting, I remember when I was still fairly new to Bitcoin I heard Taproot “introduced smart contracts to Bitcoin.” Hilarious.

All of the discrete log schemes based on finite cyclic groups of the set of integers modulo large prime p I learned in CS 407 require p to be 2048-bits! In practice a group of points over an elliptic curve with set size q of 256-bits provides similar security as AES-128. I still have to learn more, but apparently certain properties of elliptic curves allow for all the same discrete log constructions to work on these groups, and elliptic curve groups is what is used in practice on the internet today! After learning about all these constructions in CS 407 I was curious as to why Bitcoin used elliptic curve cryptography but this all makes sense now! (Beginning of chapter 15 in Boneh-Shoup textbook)

I think for a while now I’ve struggled with not knowing enough about the future of Bitcoin to be certain in my decision to work on it (or really, pursue a career in it). And a lot of my effort has been spent on trying to learn more in the hopes of gaining insight to be more confident in my decision. But I realize, you don’t have to know absolutely everything about something to want to work on it. I of course don’t know everything. I sometimes realize I haven’t realized very important aspects of Bitcoin. But that doesn’t mean I shouldn’t work on it. I find it really interesting, and it’s solving real problems. That’s compelling enough for me to want to work on it.

• Possibly even more importantly, I think no one should feel that it’s required to personally believe in the success of Bitcoin to work on it. I think it makes sense to work on it if you’re interested in helping it move forward. If anything, I want to get away from growing a stronger personally belief in Bitcoin, and just be along for the ride, searching for truth along the way.

• Meanwhile, it’s important to still form beliefs and it’s great to come to informed beliefs. I really think a fixed supply of money makes a lot of sense. I also think it makes a lot of sense that no single authority should be able to print money at will. I think it’s important for the ability to use bitcoin non-custodially to be accessible for as many people as possible. For many of these things, I don’t think we must do Bitcoin (there are probably mixed solutions that could make progress in many of the directions Bitcoin tackles), but Bitcoin does a lot of these things very well.

• I think Bitcoin is really interesting. I don’t think it will solve everything, but the things it does solve it seems to do quite well. I will continue to update my beliefs on surrounding topics. I want to be along for the ride. That’s why I want to work on Bitcoin.

I’m continuing to think about energy and Bitcoin. The more available it becomes to mine Bitcoin with excess energy produced, the more market forces will push towards the price of energy being essentially pegged to Bitcoin. My thinking: say you are an energy production company and it’s very easy to start using excess energy to mine Bitcoin. You might sell energy at some rate, but with the opportunity to mine Bitcoin, it raises that price to be at least the opportunity cost of not mining.

• At the same time, nuclear fusion might make energy super abundant anyway, so that would make energy super cheap anyway. I guess this would still bid up the required energy needed to mine profitably, if everyone is able to amass tons of energy.

• Also separately interesting, Bitcoin mining also pushes towards more effective energy production, because energy producers compete with each other to produce more energy to mine.

secp256k1 is $y^2 = x^3 + 7$. Never knew the actual equation. Can even simply visualize it on desmos.

The reason for censorship resistance.

Read on stake and consensus. My biggest takeaway from my train of thought provoked from this: in Proof of Work, as long as you are able to access some relatively random selection of nodes, you have really high assurances that you will be able to find the source of truth and it is really difficult to produce a competing source of truth. In Proof of Stake, it isn’t costly to produce another source of truth, so there may be many competing histories. When coming online, to find the right one, you have to rely on some sort of trusted group to decide what is the right chain.

• Proof of stake can probably “work” but as Poelstra says, it’s a different trust model. I’m honestly not certain on how this difference in trust model would play out over the long term and in the face of more adversaries. Nowadays, I feel like proof of work has so many benefits of being pegged with energy, and it’s tighter in terms of decentralization.

• And in general it just feels like how can you have distributed consensus on an ordering, without the notion of time?

• Links on node discovery that are relevant here: bitcoin wiki, saylor academy

ChaCha20 is Rust’s default Rng?? Cipher and RNG?

Finally looked into phantom payments. They are ways to run multiple nodes and get paid to any one of them - helps load balance payments for large volume nodes and helps with uptime if one crashes. A phantom node is one that doesn’t actually exist, but will serve as the destination for a payment where the second to last nodes in the route are real nodes that actually accept the payment. When creating an invoice the preimage and necessary information to accept a payment are communicated between the real nodes, and the sender will route through any one of them to the phantom node. Pretty cool. Can’t/don’t want to support MPP across phantom nodes because of race conditions between nodes that would be required to make payments atomic (enough). Seems mainly something that could be implemented by a user as opposed to something to support directly from LDK.

It was pretty interesting today someone was asking me about my gap semester stuff and asked if I went to Chicago for offices to work. They asked “does Bitcoin not have offices set up in Chicago or something?” And I shared a bit about how Bitcoin development works and how it gets funded, etc. This person seemed genuinely enthused at how the development of Bitcoin is decentralized, and somewhat self-sustaining, and it was cool to share that.

<C-r> in the command line will pull up a search for previous commands!](https://superuser.com/questions/299694/is-there-a-directory-history-for-bash)

I really went on quite a rabbithole today.

Cool project to port linux onto M1 macs.

Good minimal theme (honggfuzz).

Cloudflare has their own static site hosting.

Linode seems like a pretty good VPS maybe for personal usage. Cheaper than AWS and Azure, and Among Us uses it lol.

Can host your own git server with this and a web frontend with cgit or GitWeb.

FFI stands for Foreign-Function Interface and is used to bind one language to another. This is Rust’s libc.

GNU is a project to create a Unix-like operating system, that was free (as in freedom). It has a very opinionated philosophy on software being free, i.e. that computer users should have the right to see the source code, ensure it does what it says it does, change it, and distribute it freely. GNU got most of the way there, except for the kernel, which ended up being integrated with Linux, basically GNU but for the kernel, made by Linus Torvalds. People often refer to the operating system as linux, but really most of the code is actually GNU, and it’d be more apt to refer to the operating system by GNU/linux.

• Main open source licenses I feel like are important to know: GNU Public License (GPL) means that any distributed versions of this software must be open-source as well and will inherit the GPL, MIT means it’s free to change and distribute, but code using this code doesn’t have to be open-source, i.e. you can distribute as a binary, although any open-sourced code must also have the MIT license. Source. BSD two-clause is basically MIT, otherwise BSD has variations regarding a spectrum of permissions. Apache 2.0 also seems to be same as MIT. Many rust projects just do dual license Apache and MIT.

• Free vs. open source are different. People nowadays seem to say FOSS to be neutral.

Apparently Torvalds created linux because he couldn’t afford Unix. It’s probably not really true but I want it to be.

For some reason in my mind I always had this idea that Git has just always existed, but before Git people used a different version control software called Subversion which was basically where everyone just committed to main! Git introduced distributed workflows. Would’ve literally found this by visiting the home page :|.

• Git, Linux and other projects are maintained entirely over mailing lists! I remember I heard that initially Bitcoin was maintained through email, and I didn’t realize that’s an actual form of organizing development. It’s honestly crazy how oblivious I and other young developers are to where everything has come from.

Oh the internet. Visualizing commit history.

More git diff! The git documentation is honestly so good I should stop googling and look there first.

• git diff - working directory vs. staged

• git diff —cached - staged vs HEAD

• git diff HEAD - working directory vs HEAD

• git diff <commit_hash>..<commit_hash>- …

• git diff –merge-base - compares current branch (committed files..?) to where they split from each other

Now that I’ve been playing around with NeoVim some more, I’m really enjoying just exploring the kickstart.nvim file and discovering all these different things I can do and configure. Also my eyes are getting adjusted to using C-d C-u and it’s been quite helpful. ChatGPT’ing “turn this command into lua for NeoVim config” is very helpful.

NeoVim stuff:

• ' is the previous jump “register”. You can manually add to the jump list using m'. You can save a jump position with m<letter>, then go back to that position with '<letter>. Doesn’t work across files tho :(

• Add :vert to any command that opens a window to make it open via vertical split instead of horizontal.

• Learned I can configure the telescope layout, as well as open many different things with telescope through it’s different “pickers” i.e. I can map some keys to do require('telescope.builtin').lsp_implementations to open implementations of a type with telescope.

• Also learned there’s a thing called “folds” which I’m not going to look into any further but it lets you hide code temporarily and it’s built into vim.

• s deletes the current letter and goes into insert mode. Knew this had to exist.

Pretty interesting how lazy evaluation can change how you write code. I had the snippet here:

self.keys[start..end].iter().next().and_then(|key| self.map.get_key_value(key))

Which was telling me this was bad because the closure may outlive the current function and yet the closure is borrowing self and because the function’s only got a reference to self it can’t move it out of the function yada yada yada. Sometimes I just want to get to working code and so I take the compiler’s suggestions but sometimes that leads me going back and forth between solutions because the compiler is really smart but not that smart. Then I was thinking…why is the closure going to outlive this function? Which led me to discover that .and_then is lazily evaluated, and it actually moves the closure out to wherever it needs to be evaluated, which gives me all these errors. I changed my code to:

match self.keys[start..end].iter().next() {
	Some(key) => self.map.get_key_value(key),
	None => None
}

And it worked great because it was now evaluating it before moving the values out. It seems like there should be an .and_then function that would let me do exactly what I want, or maybe the compiler should know to maybe not lazily load or something (that probably doesn’t make any sense but you get the idea). But this is how it is.

Sometimes these days I catch myself being ever so subconsciously motivated by vanity metrics like my github activity chart or the quantity of my writing or even the feeling of learning by being exposed to different ideas but not really learning (like reading about something but not writing it in my own words or using what I learned in some way). Can’t let this get in the way of what’s really important! Actually learning is often slow and takes a lot of doing. Need to keep focusing on this.

history is a command that shows your recent commands. Combine with grep to find a previous commands you may have forgotten!

I want to get super good at using the command line and tools that are built into unix operating systems. How do I do this?

Idea for building a server + CLI tool with Rust: Providing educational certificates with cryptography -> applied to cryptopals!

• Summary

  • A server that will test your cryptopals solutions and gives you digitally signed certifications for each set you complete.

  • A CLI tool that makes it easy to submit to the server.

  • All code is open source. Can run the server locally and add flag --local to CLI to do everything for free.

• Usage flow

  • Run initialize command - create and save key pair to disk, send signed join message, pay invoice, receive signature

• Implementation notes

  • Pay 100 sats for 250 requests (should give you room to do the whole thing) for DoS protection

  • Identify a person by in the CLI tool allowing them to have a key pair and to make digital signatures (use PGP?) - need a good way for them to publish certification and have people verify the cert’s pubkey is actually them (use some sort of publically associated key - PGP, nostr?)

  • Verification and certification part of server can be completely stateless. Start with this. Then can build on top for a way to store previous challenge completions and certifications. Use sqlite for lightweight database. Sike, do need to store how many requests someone has made.

  • For certain challenges, it’s not just a decrypted message that can be submitted, but test cases that need to be run to prove proper completion. Maybe you will want to standardize the whole process and have them provide a command to run that will print to stdout, and so the server can send a set of random test cases and the cli tool will locally run and send the solutions.

  • Could use LDK for learning experience, or could just something like C-lightning because it’s an out of the box solution and it’s just running on a server.

• Extra features

  • Allow option to input an email as backup in case they uninstall or lose identification

  • Different pricing - 50 sats for 50 requests

Did some work with measuring memory usage of the change I’ve been exploring.

• There are many different numbers your computer can give you regarding memory, but probably the simplest and most representative of a process’s usage is just the total memory usage given by top or activity monitor (MacOS).

  • Many people online talk about using ps with rss and vsz fields, but these seem pretty inaccurate, i.e. rss (resident set size) is all the memory used to load a process’s pages but doesn’t account for shared libraries, and vsz (virtual memory size) is how much memory is allocated for the process, which is more of a maximum memory usage but not actual usage.

• Learned about the different grep’s. grep is basic regex searching, egrep is grep with metacharacters for regex such as “+*()?” etc., fgrep is no fast/fixed string grep with no regex, pgrep returns process IDs where the name/filepath matches the pattern.

• Actually used git worktree to compile test executables for my different branches all at once and it was nice. I was switching between them and had to recompile everytime, and rust compilation for tests is so slow because it cargo test will create the same test executable, and only then run that executable with your specified string for select tests. So basically it compiles all test code even if you are only doing one test. I wonder why this hasn’t been solved in rust yet because this would literally 10x every developer’s iteration speed (if they’re working in a remotely large/well-tested codebase).

A mersenne prime is a prime that is one less than a power of two.

Completed cryptopals set 3 challenge 20. Challenge 19 and 20 are breaking fixed-nonce CTR mode encryption, i.e. you are given a bunch of ciphertexts that were AES CTR mode encrypted using the same nonce. This means that the keystream that each plaintext was encrypted with is the same, meaning it’s similiar to repeated key XOR encryption, so we can break it the same way. Challenge 19 is doing this manually by guessing words and whatnot, but in challenge 20 we do it statistically by transposing the ciphertexts, and seeing what key byte would lead to the most common letters, then doing that for each byte in all ciphertexts at the same time to get the most likely key. Cool stuff. Excited to try the RNG crypto that’s up next.

Neovim approach

• Use native commands when possible

• Customize as much as I want for file exploration and things outside the main motions

• Write some of my own plugins overtime to be in full control of my editor

Today’s work agenda

• Try the changes

• Learn all that I can about benchmarking

Hmm, thinking about versioning in Bitcoin consensus versus software. In something like a programming language, a “soft-fork” is backwards compatible in the sense that it added new functionality, but any code written in a previous version will still compile and run just fine. However in Bitcoin consensus, a soft-fork is backwards compatible in the sense that all valid transactions after the fork will be considered valid by nodes running previous software, but it’s interesting to note that some transactions that were previously valid will no longer be valid after the soft fork, i.e. consensus is constrained in a soft-fork. In consensus, a new “feature” is a new constraint, whereas in a programming language a new feature is an addition.

Cool blog post - measuring memory usage in rust (rust-analyzer blog). Talks about a couple of approaches: measuring usage by data structure (heap parsing), measuring calls to alloc/dealloc, etc. However parsing heap bytes at runtime is impossible in rust because there’s no garbage collector (don’t fully understand but it sorta makes sense). However you can apparently get the total allocated memory at runtime so they measure the displacement before and after a data structure is used (or something like that..).

git worktree - seems like it’s basically just branches are in different folders in a “bare” repository? This would’ve actually been pretty helpful today when I was trying to work on two branches at the same time as well as run benchmarks on each and it was annoying to switch back and forth and recompile everytime.

Learned today people call hashtag ‘#’ octothorp.

The real thing that’s compelling about NeoVim: the skill ceiling seems very high, and it seems like I could learn a lot along the way.

• Learned a bunch of stuff that I probably can’t document all here. One cool thing was how neovim does autocomplete. There’s an autocomplete engine called nvim-cmp and you provide it with sources like your LSP or others which are separate plugins. In a vim git integration, you can even make requests to github’s API to be able to autocomplete the # PR/issue numbers while making a PR all from the command line.

• Also learned vim has some builtin autocomplete stuff with already which is pretty crazy.

This is awesome: lightning.store

Forgot what it was called: opendime

Honestly, I don’t need to use NeoVim. I like helix. I should just make my own theme.

• I like vim bindings a little more I think, and I like the universality of vim. But if I even want to migrate, I will need to configure most of the defaults of helix in neovim…

My focuses in the near future

• Continue to mature as a developer, absorb the codebase, help where I can.

• Eventually come to informed understandings of what is important for the codebase and protocol etc.

Interesting how rust implements ranges: basically just denotes an inclusive/exclusive starting and ending point, with shorthand of ... And set data structures can return a range of keys/items based on their values denoted with a range even if the full range isn’t present.

Ohhhh that’s what a short_channel_id is…block height, tx index, output index of the funding tx in 8 bytes. Human readable format is decimal conversions of the three fields separated by x’s.

Spent like 2-3 hours making my own theme in helix. It doesn’t feel like there’s a great way to develop a theme, i.e. if only there were hot reload haha. In the end there are still several things to customize and I feel like it’s still sort of half baked so I’m just using my other default theme I was already using. I’ve got some newfound respect for the theme builders.

Learned about routing gossip query messages. Nodes can query for gossip messages by short_channel_id, by block range, or by timestamp. It’s important to note that by needing to support querying by block range, the implementation will want to store some sorted order of gossip messages. I was confused when just looking at the implementation today because I didn’t realize this was part of the spec.

Stubbing in programming often means implementing a mock version of something for testing, or to make a thin wrapper for testing (say, in rust-lightning, the debug_sync crate has wrapped Mutex and RwLock for testing lock orders).

I do believe if you can’t explain something well, you don’t really understand it. That being said, being able to explain on the spot in conversation is a further level. You can really understand something, but fail to explain something on the spot, although if you can explain it on the spot you likely really understand it.

In transactions over the internet, there is always some sort of trust involved. Often uncertainties between the buyer and merchant can be resolved by in-person transactions, but over the internet you don’t have these assurances. So in some way or another you will need to trust your counterparty. However, the important thing is with Bitcoin you remove the trust in the intermediaries processing your payments, who in the current system control the payment and your funds.

Very insightful tweet

The primary importance of non-reversible payments is that you no longer need custodians of your money in order to make payments over the internet.

For smaller transactions, the risk as a buyer is lower, and the incentive for a merchant to rip off a buyer is lower. A merchant is incentivized to follow through on a transaction to build trust/reputation in order to continue doing business.

For larger transactions, or first time purchases from a seller, say on something like eBay, escrow mechanisms can be used. Note that this does reintroduce a trusted third party, but importantly the third party is not in control of the funds, but assists in mediating disputes.

Also note: for many payments, an escrow is likely not necessary, either from merchants being trusted, or transaction sizes being very low. Bitcoin enables these non-reversible transactions. This is notable, because this reduces transaction costs, allows for micropayments, and no longer requires a merchant to know much about a buyer.

• Something I haven’t dug into that much: what does it mean for the merchant to need to know a lot about the buyer? Like the fact that whenever purchasing online I have to put my name address etc. or my bank/credit card provider has to know tons of data about me? Is all of that a part of it?

Some basic resources I’ve been looking at:

• https://www.investopedia.com/articles/forex/042215/bitcoin-transactions-vs-credit-card-transactions.asp

• https://bitcointalk.org/index.php?topic=5122175

• https://bitcoin.org/bitcoin.pdf (introduction)

Working on some personal stuff.

Cool static site generator in Rust. Cool minimal theme.

Would like to learn more about taproot this week.

Hmm, apparently there are so many zero-base-fee routing nodes these days that it made sense to use just dijkstra’s instead of A*. Interesting to make this choice, i.e. the state of the network changed, so the implementation details changed. It makes sense that if it’s looking like this is the case for the foreseeable, it helps maintainability and A* could be added back in the future if needed, but it’s just interesting to see.

Gotta review my understanding of credit:

Conversation is difficult because I don’t always have every relevant fact loaded into my mental RAM to properly discuss. On the other hand, if you do, it can be an amazing interaction to surface relevant information and for people to learn and follow their curiosity. Had a conversation on credit, so wanted to review:

• I realized I misunderstood how credit card companies reverse payments. I always thought that the credit card company took the loss, and was able to make up for it from taking a cut of each transaction (taken from merchant mainly). However I realized the credit card company actually takes the funds from the merchant, because well, they can, i.e. they are basically in full control of the money in this transaction.

  • For the most part, this is probably used in situations where the payment was not made by the rightful owner, so it’s seems mainly helpful.

  • The “need” for reversible transactions however seems to be a product of insecure design of current digital payments. Every time you pay for something, you are giving away all the necessary information to spend your funds, of course that is really insecure and prone to having your information or funds stolen. In my conversation it was assumed that there was no alternative to this and so reversible transactions was a necessity to protect against fraud, but even just changing current systems to use digital signatures (or virtual card numbers) would greatly reduce the need for reversible transactions.

  • Reversible transactions are not bad, but have flaws. They require a trusted third party, which raises transactions costs—either effectively raising prices, or hurting the merchant. This eliminates the possibility for micropayments, as well as “broader cost in the loss of ability to make non-reversible payments for non-reversible services.”

    • I’m realizing I honestly don’t understand this really crucial aspect of bitcoin very deeply.

  • I do wonder the costs that are incurred on the merchant between the time the payment was made and when it was reversed, and also how often reversed payments are used in situations that are just ripping off the merchant. I know a lot of people talk about how being able to reverse a payment “screws over the merchant” but I’m wondering how direct that is meant, or it’s more about all the transaction costs involved.

• In a savings account, a bank is able to loan your money (and so you can earn interest) as they can assume you are letting them keep your money for a longer time. In a checking account, they don’t loan your money, or at least I don’t think so/hope not (and that’s why you don’t earn interest).

• I still really don’t understand why using a credit card impacts your credit worthiness.

• Interesting clip on credit as foundation of payments: https://youtu.be/NpuPm1FsUQk?t=1669, starting point → 30:02. This is really interesting to consider why people would be unbanked in the first place, although I don’t totally understand the logically all the way through.

• Main things with credit:

  • Need for reversible payments: you only need to reverse if your system is insecure

  • Reversible payments require trusted third parties - reliable third parties/financial institutions aren’t available in many places

  • Can’t do micropayments

  • Requires unnecessary information

There is so much to understand about the financial system. Sometimes I feel like just resigning from trying to understand all that I can about the financial system and just being a programmer and not worrying about anything else. There is still a lot of noise even within the bitcoin community and it’s hard to come to really informed understandings of it all. I don’t want to spread misinformation, or misrepresent Bitcoin, or blindly follow other opinions in the community. I want to keep learning and reforming my own thoughts. All I can do is try!

Spending time with friends.

Updated website.

Looking into neovim…

Motion canvas is cool! Maybe I can make some animations to help with my Bitcoin talks later in the semester.

Finished cryptopals challenges 17 and 18. 17 was the famous CBC padding oracle attack which I’d done in my computer security class, also after set 2 I feel so familiar with these CBC problems it was really not that hard. 18 was just implementing counter mode for AES and it was pretty elegant.

So it seems automated retries are necessary for trampoline. But I’m wondering, could a user have implemented that themselves before..? Regardless it makes sense for LDK to supoort it internally instead of making users do that.

Reviewed another PR!

Match guards - add conditions to match

Bunch of rust concepts!

• Unsafe just means compiler isn’t checking for certain things like needing mutable references to modify things. It’s actually common to use unsafe rust for optimizations when you can prove soundness of it’s usage.

• Monads. Still don’t totally understand the more theoretical functional programming side of things, but I learned to look for concise helpful functions on wrapped types like Option and Result to avoid having to unwrap to do stuff.

• Rayon for some helpful parallelism tools.

• &impl Trait should be the default when taking a type that implements a trait as a parameter. Often people will use generics with trait constraints, but these are less readable.

• Be careful with Deref! You can call a method of the inner type on a Arc<_> just using the normal dot notation because it implements Deref. However note: if the inner type implements something like Clone, calling .clone() will call the inner type’s clone method, and instead you may want to use Arc::clone(_)!

Pretty cool, Iter<Option<_>>::map will map to another Option, so you don’t have to handle unwrapping in order to convert the inner type.

Updated my followup PR.

Probably going to do some review and cryptopals today.

Considering switching to NeoVim…Vim is just more universal and I imagine everything I do in helix can be done in NeoVim. Then again, it is really nice how small the config is, and it’s not like I’m working on remote servers all that often…

Feeling like I’m getting a better foundation for how to write good PRs/just be a better open-source contributor/software developer. Getting better at manipulating commits, understanding better when it makes sense to separate commits and whether to squash fixups and how to lookover my changes before pushing.

Realized payment_secret doesn’t really need to tie together MPP HTLCs because they also all use the same payment_hash.

Reviewing

• Why is it necessary to move payment retrying for trampoline?

  • Saw in preceding PRs that moving payment retrying into ChannelManager is to be able to support trampoline payments. Is this to support retrying payments when a node built with LDK is acting as a trampoline node (did automated retrying not exist in LDK before)?

• Dang so, you don’t need a mutable reference to an object if its behind a mutex (outbound_payment::abandon_payment, check_retry_payments, etc.)

  • So apparently there’s “interior mutability” which basically breaks the core laws of Rust wtf!! Sometimes you want to have a shared reference that’s mutable by multiple parties (Mutex, doesn’t differentiate between readers and writers), and sometimes you do want to have a shared reference where different parties are reading or writing (RwLock, one writer many readers), along with other instances where you need to employ mutability where Rust typically wouldn’t let you.

  • What is the reason for changing abandon_payment and check_retry_payments to take in pending_events as opposed to how it was before. Is it mainly just because it’s a bit cleaner or is there other significance?

I love Bitcoin Optech: great summary on trampoline.

• Question: to conceal whether a trampoline node is sending to the final destination, the recipient must also be a trampoline node right (which definitely wouldn’t always be the case)? Because in order for it to be ambiguous, a trampoline would have to not be able to tell if it was sending to another trampoline or not, but whether nodes support trampoline is public or easily discovered right?

In the discord, someone asked about exposing the ip:port info of connected peers, and someone said that they didn’t even keep that info until recently. My question: how do nodes communicate without ip:port???

• It’s not magic lol, LDK just doesn’t keep the state itself, it holds the abstracted TCP connection (which uses ip:port on a lower level) and has the option to provide ip:port to LDK but doesn’t have to

Learned about interactive staging/commiting with git add -p and git commit -p. Pretty cool if you want to split up a commit, or only stage certain changes.

PR was merged! This one took < 24 hours, whereas my first one took ~2 weeks. Lesson: look for good issues, i.e. ones that are straightforward, but more importantly very directly helpful.

Opened this followup PR.

Learned about ZFR (Zero Fee Routing). A guy did an experiment where he ran a huge node where he allowed zero fee routing, although he would charge a fee to open a channel with him. Super cool.

• During the interview, when asked “If there was one thing you could change about the lightning protocol, what would it be?” he answered with the ability to overpay a MPP, i.e. to pay a 10 sat over MPP, send 1 sat over 15 paths, and cancel the other paths after 10 have been received.

• This interview was only 3 months ago, so I’m assuming it’s not implemented. Would be cool to develop this, because it makes a lot of sense that it could really help success rate of MPP for large nodes.

Looking into MPP. Confused right now what is the point of payment_secret? And how are MPP tied together? Are they atomic (i.e. either all is paid or none is paid)?

• Something important I’m understanding: how probing works is that some intermediate hop on a route can probe for final nodes by adding an HTLC with that node and checking the response (or at least something like this…?). It knows the right payment hash and the amount which is (basically..) enough to pretend to be the last intermediate node. payment_secret is another field that is included in the invoice + final node’s onion payload that verifies that the payment/HTLC was actually sent from the intended sender.

• payment_secret is also said to “tie MPP HTLCs together.” I guess does that just verify that different payments in an MPP all came from the same recipient?

• From what I can tell, only Base AMP (idk why it’s Amp when it’s not atomic) is implemented in the current spec, i.e. MPP are not atomic…but at least partial payments are incentivized against…? Confirmed they are not “truly” atomic!

• Questions for the ldk discord:

  • Payment secret verification: I’ve got a clarification question on payment_secret:

    • Because it’s communicated through an invoice, do keysend payments not have them? And does it mean that keysend payments are subject to probing attacks that payment_secret prevents?

      • Answer: the preimage in the keysend payments serves the same purpose!!

    • When it’s said to tie together MPP HTLCs, does that just mean the final node identifies that all the partial payments are connected because they have the same secret, i.e. were sent by the same sender?

      • Answer: yes

  • Do nodes currently only try routes one at a time? Is there a way to overpay/try multiple routes at the same time?

    • Answer: there is a proposal for this somewhere! although likely relies on onion messages

Lightning-dev mailing list page. Interesting it used to be called a “caching layer”

Got a bit ahead of myself getting excited about possibly trying to develop the “overpaying” feature that would increase routing success rate. Realized I should really just focus on the problems we face currently, because there’s really still a lot to do.

Can easily derive std::error::Error trait on a custom error with thiserror crate

Feeling pumped to get back to contributing this week. I think I’m doing a good job of finding balance between code review, making PRs, and building with/using the software (even though I’ve only done a little of each).

Helix: space d for diagnostic picker, great for going straight to compilation errors.

• Would be nice to have a “change picker” in helix, so I could easily cycle through all my changes. Probably something exists in git I just don’t know how.

Use single quotes for git commit messages to avoid shell execution of backticks

Cargo has built in benchmarking with cargo bench! (also done in CI)…but I don’t get the nice time tables…

New thing to look out for when making a PR: fuzzing. Command: cd fuzz && RUSTFLAGS="--cfg=fuzzing" cargo test --verbose --color always

Got through my PR and it has a 6x speedup for reading the network graph! It was pretty simple though, the heavy lifting was already done a while ago hehe. I think the real value was just recognizing that there was potential for speed up here (which again was not done by me lol). Nevertheless, feels good to help out!

• Note to self: look for more issues like this! I can still learn from them, and I can maximize my effort:being helpful ratio.

Working on cryptopals challenge 16 and I’m sort of curious about how this would work in a real context, because I imagine they have some sort of other santization/checks on the decrypted plaintext before returning the padding error. I guess not? Or at least, in the famous case they didn’t (these days I feel like it’d be caught much more easily).

• Finished all of cryptopals set 2 and excited for set 3!

It’s interesting the balance of having a strict and lax filter for content/ideas. I’ve recently been intaking more stuff by Peter Zeihan who wrote this popular book lately. On one hand, he’s gotten me to think and understand connections I would get from few other places, and even before I’d known his background, I just thought he was some random guy, and having that low barrier for considering his ideas was valuable. But at the same time, he seems to just talk with little consideration, as in he states things as fact that he might not necessarily know for sure (and seldom acknowledges his uncertainty), e.g. he said something about blockchain the other day that was really stupid, and it made me question everything else he’s said. If I were to filter out all of the kinds of content that do something like this, I may be led to read only the greatest texts and go directly to the source, which would maybe be worthwhile in its own right.

Building with LDK

• I’ve realized that ldk-sample > getting started docs. It’s kept up to date very well.

  • I’ve also realized few people are probably going to start building a node or something from scratch, and instead will fork ldk-sample. I got a little bit of an idea of what are the components that come together from trying this out, but it wasn’t as helpful as I thought. I’m still sort of aimless in what I want to build, which I think has been holding me back.

  • Idea: turn ldk-sample into a daemon + CLI, integrate with a BDK wallet, use BIP 157/158 as chain source (contribute to BDK?), use rapid gossip sync, figure out async payments, slap it on mobile

    • End goal: self-custodial lightning on mobile. If I’m building something let’s just go for that out the gate and see where it takes us

  • I think I will have better luck building and learning if I just try to adapt ldk-sample to what I want to do instead of building from scratch. In the meantime I’ll pick up another issue because I don’t think I’m going to make too much progress building something on my own at this very moment, but I can help out on smaller stuff in the meantime.

Attack surface: I saw there is a setting for manually_accept_inbound_channels and the default is false. It made sense to me that it’d be false, because what harm is there in having someone provide you with inbound liquidity and pay to open a channel? But if it’s someone you don’t know/trust, you may end up using this channel to pay or be paid. If you’re not careful, just from someone having a channel open with you could lead to: channel jamming by routing through you, involving you in an attack where they force many channel parties to have to broadcast on-chain at the same time, they could making routing more difficult if they know you’re relying on them, probably more things.

• A bit ago when I gave the part 2 of my Bitcoin talk, someone asked “doesn’t lightning reintroduce that reliance of trust” getting at that it’s contradictory to the ethos of Bitcoin. I answered saying how lightning is designed to still be trust minimizing, and at the time I felt that on aggregate lightning was still pretty much trustless, that at the end of the day whatever your counterparty does, you can always get your funds back. But it’s more nuanced than that. First, it’s not totally trustless (much more than fiat, but much less than on-chain payments) (at the moment), it definitely does bring a bit of trust back in, but importantly, that’s a part of how a layered money system works. It brings some trust back in, but allows for fast and cheap payments.

Trying pick up this issue. Looks fun.

Posted my cryptopals implementations so far if people want to check them out. Would be cool to eventually refactor this repo to be just tests with empty functions for people to implement. I feel like making the tests yourself is kind of part of the exercise but it’d also be nice to not have to.

Yes, electrs built! See https://github.com/romanz/electrs/issues/753 for M1

Installed electrs.

Also briefly tested by electrs setup with bdk-cli.

My command for electrs:

./target/release/electrs --network=regtest --daemon-dir=/Users/alecchen/Library/Application\ Support/Bitcoin --log-filters INFO

My command to run bitcoind for electrs (pretty basic):

bitcoind -regtest -server -daemon

It was great to set up my own electrs server, but I think nigiri is winning out for my development setup.

Building with LDK:

• Filter::register_output function signature no longer has return type
• The filter section is not super clear, or at least I am unsure what to do just from the docs that are here.

Great article on why Breez is using Neutrino + choice of full node. Great article comparing all the trust models for wallets.

• Honestly can’t really make up my mind on the matter. Generally however, it seems BIP 157/158 clients are better than Electrum, because Electrum is essentially just SPV. It seems to me all of these light clients suffer from the same issues of just not being a full node and validating consensus, and vary more subtly on privacy etc. So you really just want to make sure that you are connected to an honest full node. Only way to really guarentee that is either by running your own node (not available for certain resource contrained circumstances) or by connecting to different full nodes/servers.

Completed cryptopals challenge 14. 15 and 16 don’t look too bad. Almost done with set 2!

Haven’t checked the chaincode slack in a while - stumbled upon this search tool which is hella cool.

Watched Ray Dalio’s 30-minute summary on economics.

• Lovely animations.

• Super interesting how he defined settlement - when you buy credit (debt), and then pay it back. Good definition imo.

• Good note on how credit is good when it can finance a business expense that leads to higher productivity, bad when it leads to overconsumption that you can’t pay back.

  • Which one do we have more of in our country? (consumption) Does one require the other? (no!)

• The way the central bank was described: it controls the economy. Yikes.

• The way that credit/debt is used in order to reduce uncertainty in the future, i.e. when you spend more now, you must produce more later. I can sort of understand that you could say it’s good to influence people into making themselves produce more, but it sounds sort of manipulative in my opinion. Don’t know if it’s a problem with education, and/or a problem with top-down normalization of economic behavior, but it seems manipulative in how this trend is imposed on people today. Personally feel like producing up front and saving leads to more freedom/well-being..

• It honestly just seemed like all of the ways the economy is manipulated aren’t that necessary. Accumulating tons of debt leading to increased spending and income, and then having to very carefully manage the recession sounds like unnecessary indulgence leading to risk of serious economic suffering. Like it sounds like the government is eating a bunch of candy, getting diabetes, then supplementing insulin. What I mean is it’s causing its own problems. It seems like debt doesn’t even necessary lead to productivity growth, it just leads to fluctuation and risk.

• You can have productivity growth without debt cycles.

Went through OptionalField vs Option again. Found where many different types implement the Writeable/ Readable serialization traits which brought some clarity. Kinda confused with how the spec differs from the LDK implementation in that in open_channel, accept_channel and I think one other message, the spec makes it look like the way shutdown_script is serialized has changed, but LDK does not reflect that. Asked this in response to a comment on Github so hopefully I’ll get the answer.

Ended up cleaning up a script for the Illini Blockchain developer take home. console.table is pretty nice.

Didn’t get to much today, sigh.

Hooray, no headache this morning!

Back to trying to build. I think I’ll see how far I get by the end of the week and decide whether I’ll try to make another PR or keep trying to build using LDK.

• I feel like building using LDK at the same time as contributing to LDK is the ideal combo if I can manage the time. It would be cool to just build a phoenix-like self-custodial wallet with LDK + BDK.

Building with LDK

• Logging is clean with macros!

• Next: get electrs running or use esplora and reimplement getting fee estimates and broadcasting

Helix (idk about other editors) cannot easily let you bounce using LSP stuff in two project directories at the same time, i.e. I tried to have both ldk-sample and my project open and it could only do LSP rendering for one.

• Also you have to actually import or use a file for helix LSP to turn on, i.e. I tried to just put my file in the ldk-sample project directory, but I needed to mod other_mod; to use them, which messed up imports and was annoying. Have resorted to just having helix up in separate terminal windows.

Can definitely start doing more review that’s not just public documentation - asking more questions about how something is done or just about the area of a codebase. Should get so at least one more review like this by end of tomorrow.

Pretty fire video on Rust smart pointers - allocating on the heap intelligently. Ideally would like everything to be on the stack because it has a faster access time, however to be on the stack the size of the object must be known at compile time, which is not always the case.

• Box - basically just a pointer/borrowed reference to an object. Used for when you have a trait object who’s type can’t be known at compile time, OR just when you need indirection for something like a recursive data type, e.g. self-referential struct

• Rc - Reference counter. Used for when you have multiple references to an object allocated on the heap, and don’t want it to go out of scope until all those references are done being used.

• Arc - Atomic reference counter. When you need an Rc to be thread-safe.

Finally getting some clarity on how Bitcoin wallets work and how they’re developed.

• Electrum is a server and SPV client/wallet. The server is useful to index block data in a way that you can easily retrieve what’s needed for an SPV wallet. electrs is a re-implementation of Electrum server to be able to self host an electrum server with not much more resources than a full node. Blockstream has a fork that’s more powerful and better for public use: Esplora, which includes an HTTP API on top of electrum.

• BDK has an electrum client implementation here.

• BDK primarily supports electrum and esplora for getting blockchain data

• Full node vs SPV wallet: SPV wallets only check that a transaction is included in a block, whereas full nodes enforce consensus no matter what. Main point: both will follow the majority of miners in the case an attacker overpowers the network, however if miners were to change consensus and fork (provided these miners had majority hashpower, say to increase block reward) SPV nodes would blindly follow the consensus change (to add onto this, there could be completely faulty transactions without signatures etc. that light clients don’t check for), whereas full nodes would maintain whatever consensus they were already validating. “It is only reasonably secure to use a lightweight node because most of the Bitcoin [network] uses full nodes.”

• Nigiri - spins up bitcoin core and electrum server in a docker container

Dang this is cool, USC students built a bitcoin wallet using BDK in fall 2020

Super distracted today from messages and whatnot, need to ignore my phone tomorrow.

Had a migraine for the first time this morning. Very painful. Taking it easy today.

Did cryptopals challenge 13. These challenges are often not super clear in their directions…I guess that’s sorta the point haha. Because we’re writing our own tests and whatnot, it’s sometimes just not clear exactly what we’re trying to do or what the context is, but eventually I end up figuring it out.

Praying my headache resolves tomorrow morning!

Onion messages

• Main question: so the point of route blinding is to conceal the recipient’s info right. What information is there to conceal when routing just a normal onion message, like an invoice request? To provide their half of the blinded route, the recipient has to directly communicate with the sender, so why does the sender have to route the onion message?

  • However for something like an invoice request, what’s the point of routing that?

• What is the point of routing a message if there is no state that needs to be kept (e.g. updating HTLCs) along the route?

• They include a blinded route for the recipient to reply with??

• If recipient reveals how many hops they are from the blinding point node, after a couple messages to the same person, would they reveal who they are? Knowing that you’re 2-3 hops from two distinct points could narrow down the options quite a bit if an attacker is looking for it? (I guess how would they know it’s from the same person?)

• To discord: Confused about onion messages/blinded routes: what is the reason for creating a blinded route for a message when a recipient already has to directly communicate to the sender to send their blinded path? In that case why doesn’t the sender just send the message directly? I’m mainly thinking of something like an invoice request or some message that doesn’t really concern the nodes it’s routed through

Ok got some clarification:

• blinded routes are communicated over a QR code or some other communication channel outside the lightning network, so the recipient is not directly communicating with the sender, and thus the recipient’s identity and nearby peers are concealed. note: “peers” here means just anyone a lightning node is connected to for communication is general, messages in general (i.e. gossip) are forwarded along these normally.

• In the future, to prevent people spamming onion messages, these peers will likely be limited to channel peers + some sort of rate limiting.

Got some feedback from review on PR - rust macro pat identifier pretty cool

Played around with Phoenix wallet today. Really impressed with the UX. Besides having to understand a bit about lightning, the UX was honestly amazing.

Building with LDK:

• Let’s just build a node and see what I run into.

• For on-chain wallet, let’s use BDK, and for on-chain data let’s use bitcoin core + electrum server?

• Build from scratch but have ldk-sample open for when you want to use default implementations such as FileSystemLogger

• Stuff I’m running into

  • Doc edit: byte[] in https://lightningdevkit.org/tutorials/build_a_node_in_rust/#step-9-sync-channelmonitors-and-channelmanager-to-chain-tip

Rewatching this:

• Always-online problem has a bunch of tricks but none really solve it completely because they usually rely on the fact that you end up being online fairly frequently, but that is not always going to work. Async payments basic description: node tells sender when they’re online. Look into PTLCs!

• Watchtowers don’t work for HTLCs (i.e. if someone delays and broadcasts an HTLC to the chain, a watchtower would need to know about your upstream/downstream channels + any payments you’re routing in order to redeem the payments by watching on-chain for a preimage but they don’t do that) they are only for revocation punishment

• The data required for a watchtower (or just node needing to punish revocation) grows unbounded as a channel continues forward. Eltoo solves this by only requiring a constant size piece to data.

• Review: anchor outputs are to allow child-pays-for-parent to update fees when channel counterparty disappears.

  • Enter pinning attacks: your channel counterparty broadcasts their commitment transaction with CPFP but with low fees, and you’d like to broadcast your commitment transaction with higher fees but Bitcoin core isn’t smart enough to realize to replace this (yet…Gloria’s work fixes this!).

  • Sike! There’s another problem. As an anti-DoS limit, Bitcoin Core has a concept of “no free relay” meaning you have to pay 1 satoshi/vByte to replace a transaction, essentially paying for the validation compute you incurred. This means your counterparty can broadcast an enormous child transaction, screwing you over.

    • Gloria’s work fixes this again! Special tx v3 basically says this package (connected txs) is able to be replaced if it’s in this specified format.

• Lightning currently (and eltoo) relies on the fact that you will be able to broadcast a transaction on-chain within a certain period of time. But block space is scarce, what if everyone’s broadcasting at the same time? Some people must lose money. This is a long-term concern, no great solution for it currently. For the time being, don’t open/use channels with people you don’t somewhat trust.

• Routing is hard. In discussion whether rebalancing is zero-sum (just pushes payment towards other people and makes it hard for them to route) and if payment flows are inevitably difficult and need some other solution.

  • Good amount of nodes on the network (club net smth..?) are not well maintained and make routing hard as well.

• Privacy is hard because clustering is probably going to beat everything you have, and even if many of your off-chain payments are unknown once you broadcast then clustering will find your utxos again and more.

• Channel jamming is also a thing: someone sends a bunch of HTLCs, and just waits for their expiry - locks up liquidity to cause more difficulty routing (or even prevent entirely for some time). Great research done recently into this, but not resolved yet.

Original Lightning Network paper had some pretty interesting estimates concerning scalability. In section 10 “Block Size Increases and Consensus” it estimates

“If we presume that a decentralized payment network exists and one user will make 3 blockchain transactions per year on average, Bitcoin will be able to support over 35 million users with 1MB blocks in ideal circumstances (assuming 2000 transactions/MB, or 500 bytes/Tx)

• Note: transaction sizes have decreased (especially multisig transactions important for lightning), and segwit increased block size. Although there still seems to be some unanswered questions here (at least from my current knowledge).

• Interesting article on transaction sizes. Pretty quality article I thought, oddly has a bunch of kind of sus links at the end..

Helix: space s opens symbol picker, which is basically any notable type/field/function/etc. that shows up in the current file, space h selects symbol references within current scope so you can easily edit at all of them or just see them, space r directly lets you rename

Reviewed another PR!

Everyday I understand more, I also cringe more about times I’d talked to someone and given a misinformed answer. Nevertheless, just gotta keep learning more and doing my best!

Money plays such a big role in our lives because we are dependent on each other, i.e. society. This dependency is involuntary, but also greatly to our benefit. People end up feeling like a slave to this system, to earning money, because they view it in the context of having so many fundamental problems of survival having been solved before they were even born. You could be independent of the larger society, maybe live somewhere remote, but you would have to: build your own housing, source your own water, boil/clean it, grow gather or hunt your own food, cook and preserve that food, protect yourself from disease, create your own clothes, tools, and many more, not to mention all the uncertainty that accompanies them.

• Today we have different problems, and all now as important to solve as our other problems were previously. But let’s understand in the developed world we take so much for granted! We are born freed from so many burdens!

• That being said, there are many people that are at the mercy of their economic situation—that is not to be left unrecognized. However I don’t think it’s money that’s the source of many of these problems, but the restrictions/distortions/influences on people’s ability to make and save money themselves, as well as education among other things.

I think it will be fun and insightful to build something using LDK. Can keep reviewing PRs and asking questions in the discord in the meantime. Right now I can be sort of helpful because I’m a new perspective, but my perspective is basically some random person who knows a little about lightning and bitcoin. I think I can probably be more helpful by trying to use the software. Then, I’ll be able to provide insight for one of their main audiences: people trying get started building in lightning.

Also want to read more from the source on economics to come to my own conclusions. Wealth of nations, austrian material, keynesian material, others.

I realize I think I’m more of a low-level person. I like product/user-facing stuff, but it’s more like candy, whereas protocol development, cryptography, and systems programming are the things that are the most intrinsically fun for me.

Guy Swann episode on Stephan livera podcast

• Converge on one money because it’s a communication platform, just for value. Just like using traffic lights and paint on the road is an agreed on way of communicating how to safely navigate the street, the fact that we all use the same one means we can function properly.

• Other tokens and whatnot may have some utility—many are straight gambling—but they will never be money. The ultimate decider is what’s the most salable, and nothing will be more salable than bitcoin (in the long term). For a while I’ve been weary to say Bitcoin will be the one, like oh who knows it’s hard to predict, but with more macroeconomic perspective lately, I’m getting more clarity on the convergence to a single monetary medium.

  • Will clash of clans elixir become money in the presence of Bitcoin?

Joined and introduced myself in the Summer of Bitcoin discord. Also recommended the program to a freshman in my club that seems pretty bright/curious. Would be super cool to see him get into it and have a good experience.

Completed cryptopals challenge 11.

Football vs basketball society: strengthening your weakest link is more important than strengthening your strongest link (this is a rapid change that has occurred lately). Really cool to hear about because I’ve personally come to the conclusion on my own that at the least it’s more interesting to me to raise the floor.

Went through a git exercise with Illini Blockchain tech leads and it was bleh. Went through it on the tv screen so they could follow along, but should’ve just had them just go through it and ask for questions when they needed help. I was still a little unprepared which made it lower quality. Honestly most of them probably know this stuff just fine. Feel like I wasted their time.

Learnings from my Illini Blockchain tech lead training: git rebase master tries to rebase on my local branch. The difference between git pull and git fetch is that git fetch is safer because it gets the latest changes without making changes to your local files.

So when I fetched, my master branch didn’t have the changes. When you fetch, you get an output like

$ git fetch origin master
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), 675 bytes | 225.00 KiB/s, done.
From https://github.com/IlliniBlockchain/git-sample-alec
* branch         master -> FETCH_HEAD
f769a97..55ffe3f master -> origin/master

Which is telling you that you can reference the changes with FETCH_HEAD or origin/master. So I should’ve just done git rebase FETCH_HEAD or git rebase origin/master and everything works just fine. Honestly kinda can’t believe I didn’t know this difference until now lol.

Had a crazy dream where I was using helix keybindings to navigate maps lol

Saw in LDK discord persisting channel update monitor updates asynchronously needs to be fixed

[workspace] + members=["...",...] + exclude=["...",...] in Cargo.toml allows for you to manage multiple packages together (share dependencies, build scripts, etc. “Packages that are members of a workspace can depend on each other, and when you build or run tests for one package, Cargo will automatically build or test all the other packages in the workspace as well.” - ChatGPT

Was intimidated because I saw a lot of lines of code in this PR but most of it is documentation! Lovely!

Review:

• Look into these - they seem like prerequisites for this macro?

  use lightning::ln::peer_handler::CustomMessageHandler;
  use lightning::ln::wire::{CustomMessageReader, self};
  

• What is this PR doing?

  • Defines a macro that makes a custom message handler. This takes in custom types and as long as they have certain serialization traits defined, it can compose messages with them

  • What is a custom message? What’s it for?

• Why is a separate crate needed? read description

• Why is foo_bar_type_ids an OR of the two type ids?

• Very weird: $($field_vis:vis $field:ident: $type:ty),* just matches a comma separated list, you have to add a $(,)* at the end to include trailing commas (and it doesn’t account for extra commas which I thought would be the reason for * over ?…but it appears they do the same thing here…)

• Right now, does peermanager support multiple custom message handlers?

• Pretty cool, learned about the different frag spec keywords in rust macros

Standards for Efficient Cryptography (SEC) 1: ECC

Glossary of types used throughout BOLTs - has point and short_channel_id which I’ve gone out of my way to look for in the past

Maybe I should make my own lisp at some point

BOLT 11 invoices are dangerous to be used twice because if a preimage has been made known to anyone along the path already, they can just seize the HTLC without forwarding on the route.

BOLT 12 - offers (static invoices; not really invoices themselves but static data that can be used to request or send an invoice), new invoices, invoice requests

Onion messages (summary, BOLT PR)- I get why an HTLC needs to be onion routed—each hop needs to forward a payment in order to pay the final recipient—but I don’t get what onion messages are for. To use blinded paths for payment info, that makes sense, but onion messages are said to be a general purpose message?

PTLCs - point time locked contracts, instead of requiring preimage to unlock, require signature from point (on elliptic curve) - need to look into more to understand significance

Did more of the developer take home for Illini Blockchain—set up something pretty cool today. A large part of the task is to send us an encrypted message. Until now, applicants couldn’t really check if we got their messages correctly because they of course don’t have the private key to decrypt (other applicant submissions are public, so if anyone had the key it would reveal other applicants’ info!). Today I set up a server, where they can check if a submission starts with a 32-byte secret, and only then will it reveal the plaintext message received (and they’ll be instructed to start their message with a randomly generated secret). Deployed it on AWS and directed a subdomain to point to it which I thought was cool.

Watched conor okus and val wallace on bitcoin developers livestream today talk about how to start contributing to LDK. Main highlights: as a new perspective giving feedback on public documentation is super helpful, review and just asking questions is a great way to learn about the code base and often leads to improvements to the code, being a user and just trying to build using LDK and giving feedback is a great way to contribute

• Side note: super interesting LDK tries not to use Vecs whenever possible because of memory fragmentation.

Process for review (helping myself organize in my head):

• Pick a PR (can be anything, but particularly helpful if it’s in the current milestone target or has newly added public documentation)

• Figure out generally what it’s doing and why

• Look at how it’s doing that through the code and changes

• The previous 2 take time and lots of reading on your own, but that’s how you learn!

• Jot down any questions, and at the end if they are unanswered, leave a review with things that you were confused by

Randomly an amazing summary of free markets/price system https://youtube.com/clip/UgkxOtJHnyEPIgjsCCG05t5BFPMPWjZapZOQ

Important concepts just swirling on my mind

• Decentralized/trust-minimizing solutions will not have as good or better UX than something that makes efficient use of a trusted system. The goal is to close the gap as much as you can however. Importantly, building trust-minimizing solutions serve a different audience.

• Bitcoin’s price in USD will become less volatile when there is more widespread consensus on it’s value. It sounds obvious but for a while I just thought who knows, maybe when everyone’s already transacting with it, or maybe merchants will have to list two prices or something. But a currency transitions from collectible, to store of value, to medium of exchange, then to unit of account. It’s currently becoming more of a widespread store of value (widespread being maybe 1% of the world has exposure), and a medium of exchange for some, but that is still far away (in terms of current state of things; the amount of time it takes could be long or short)

• Bitcoin macroeconomics are not that hard to understand. I keep feeling like oh I should do more research into this, but the more I think of it, the more it just makes sense. Inflation is bad: raises time preference, it’s an unconsentual tax, disproportionately harms less fortunate, incentivizes debt and consumption, forces you to become a financial analyst or pay someone to do it for you (once again hurting less fortunate). When it comes to money, the nuanced influence on behavior adds up. Fixed supply is as scarce as you get; decreasing supply is impossible—what are you going to take people’s money? (besides I don’t really think a strictly decreasing supply helps with anything).

• Converging on a single currency just helps that currency do it’s job better. If there are multiple currencies, it’s more difficult to understand prices and communicate value, i.e. prices are slightly distorted, and on a large scale like the whole world, that can have a great impact.

• The existence of money and free markets at their core just incentivize you to earn money. It’s structures in our society (and the lack of soundness of the money itself) that lead to the pursuit of money incentivizing the wrong things. Money is not really a social construct, but a tool that emerges to help people exchange value.

Ohh, electrum is a protocol to connect to a full node (running on a server) to source chain data (and it’s also a desktop wallet?)

Responded to more feedback on my PR and it got merged! Some reflection:

• Sure I’m new so I’m not expected to know absolutely everything, but…

  • I should be more diligent about thoroughly reviewing my commits/code before putting up for others’ inspection - especially now that I have more things I know to look for

    • It’s nice to respond to feedback quickly, but I should prioritize being confident in my changes. Confidence with speed will come with time.

  • It’s a serious, heavy duty project. If I expect to contribute anything meaningful, I’m going to have to know the protocol and codebase in detail.

  • I should really understand more about how this codebase is used. If I’m going to contribute, I need to have a good idea of what it’s being used for to understand how to best implement a solution to an issue.

• Get in the habit of - running test and autoformat on every commit, squashing/rebasing when it makes sense, rebase don’t merge main, checking that the commits are what you think they are (weird changes don’t get rebased) and don’t have whitespace, making sure if you are expecting someone to review it that anytime you make a change you have reviewed the whole PR and you didn’t miss anything trivial that shouldn’t be wasting reviewers’ bandwidth, always assume unwraps are unsafe unless they are from locks

• Reviewing code is one of the best ways to start contributing.

• I sort of just want to open more PRs and get more commits in, but I think I would be able to learn more and contribute more by doing a lot more review, and taking some time before picking up another issue. I can still explore the open issues, and if there’s something that I think I understand pretty well, I can pick it up.

• I’m considering maybe even trying to build something using LDK, maybe at least something small, just to get a better understanding of the user than I get from ldk-sample. Maybe I should do more research into what wallet developers face? I don’t know, but something to get a better understanding of a user. Maybe it’ll just come with deeper understanding of the protocol and codebase.

Finished the saylor course certifications for summer of bitcoin screening today

It’s cool, I realize I don’t have to (and really shouldn’t) try to follow some regiment to try and force myself to get better at lightning stuff. I can just seek out whatever interests me, and that will most likely naturally lead to me learning more. I need to knudge myself to remember to look at more code and to really go in depth, but otherwise I can jump around as I please.

Reviewed the Dual funding PR!

Learned a little bit about discrete log contracts. Use a trusted third party oracle as a way of getting external data on-chain in order to do “real” contracts.

Got my question answered about the weird failure code stuff in BOLT 4. The certain probing attack in the failure messages section was more relevant (specifically to LND) before payment secrets became a thing. Surprised this even got merged because it seems like more of an implementation issue.

Learned this a bit ago but forgot to include it in a dev log: Payment secret was added as something to validate MPP. The payment hash is communicated to each node along the route (in order to form correct HTLCs), but the payment secret is only known by the sender and receiver and is used to validate the payment, and can be “split up” for MPP.

A point type when talked about in the BOLTs corresponds to a public key. Kinda confused now, but I guess they’re all just 32 byte numbers so..?

Plan for today

• Attend LDK sync - prob don’t need to be apart of this since I’m not participating and there are meeting notes. Cool to sit in once though.
• Review a PR - maybe Dual funding, maybe the other documentation one, conveniently this has a lot to do with channel opening which was I diving into on Saturday!

Dual funding: mainly a new generic channel establishment protocol, helpful to those wanting to open a channel to receive payments (previously can’t open a channel just to receive, in case attackers spam making people pay on-chain fees and running away), helpful for dual funding a channel to send and receive, can be combined with liquidity ads to help liquidity buyers and sellers find each other.

• This is mainly just a change to how the funding transaction is created. Instead of the channel funder creating the tx on their own and immediately jumping to sending commitment tx signatures, they build the funding tx together through a sequence of back and forth messages called interactive-tx.

Funky: nSequence. Originally intended to be used for RBF—miners would prioritize transactions with higher sequence numbers—but did not end up being used like this because you can’t really enforce profit-seeking miners’ behavior. In BIP 68 using this for transaction-level delay was implemented (spec has a great diagram). It allowed for reinterpreting the nSequence number for relative time locks, importantly enabling HTLCs. OP_CSV accompanied this in BIP 112.

• Funky part is how nSequence is an input level parameter, but it is really only meant to be used on a transaction level, i.e. OP_CSV checks the nSequence of the input. ISTM if multiple inputs have a sequence number, then the max is taken.

• nLocktime is a transaction level parameter, and seems to work slightly differently. nLocktime on its own seems like it can just be used to pre-sign transactions that can’t be mined before some block height, but it can also be used similar to nSequence where it’s used on a script level by OP_CLTV where the output can’t be spent until that time.

• My question: why don’t you just specify the number in the script for OP_CSV or OP_CLTV instead of using the transaction/input fields?

• Wow, this resource helped a lot. nSequence is really for relative to when a previous transaction was mined, then OP_CSV is script-level relative to when this transaction/input is mined.

A basepoint is a point on an elliptic curve (G) that gets multiplied by a private key (n) to form a public key (nG). A little confused on how the cryptography works for there being many basepoints on a curve and how you’re able to generate multiple public keys from the same private key. (all the basepoints involved in commitment txs, also HD wallets).

Secp256k1 and ed25519 are both elliptic curves. Bitcoin uses Secp256k1. It’s better tested (libsecp256k1) and smth smth prime number of points, no low order points.

ChaCha20 is a symmetric encryption scheme, faster than AES.

If you want to combine two if let statements, you can just put a comma in between:

if let Some(x) = x_wrap, let Some(y) = y_wrap {}

More helix: removed overwrite of space w because on it’s own it switches to window mode which is way better than C-w. space y yanks to system clipboard! Set idle-timeout to 0 so autocomplete is faster!

More notes from saylor stuff for summer of bitcoin assignment:

• Saylor on Stephen livera podcast

  • Inflation is a vector. Propoganda: we can’t change what people think, but we can change what people think about. CPI inflation is misleading, can be chosen and changed by few to support what they want people to think.

• Bitcoin obsoletes all other money

  • We naturally converge on a single (the best) money because it’s in our best interest. Finite scarcity is powerful.

  • Converging on a single currency is important because it drastically improves the currency’s function as money - price system is less distorted → incentives are communicated more effectively, economic resources are utilized to their best potential.

  • “I don’t believe we shall ever have a good money again before we take the thing out of the hands of government, that is, we can’t take it violently out of the hands of government, all we can do is by some sly roundabout way introduce something that they can’t stop”. – F.A. Hayek.

    • Haha, no way

Asked a question about failure messages in the LDK discord

Some notes from Saylor academy:

• The bullish case for bitcoin

  • 4 stages of a monetary good serving as money: collectible, store of value, medium of exchange, unit of account

  • The price is uncorrelated to its utility, but just to its adoption (which you could argue is it’s utility) - generally it seems like a bubble, but monetary goods are just priced very differently.

  • After there is wide consensus of Bitcoin as a store of value, volatility decreases and makes it easier to use as a medium of exchange.

New helix things: space+a pulls up actions on diagnostics, in file picker C-v opens and vertical splits

How LDK does payment preimages: the user can supply a hash, and then they’re responsible for storing and providing it, however if they don’t supply it, LDK can create one. I noticed the return value for their function create_inbound_payment only had Result<(PaymentHash, PaymentSecret). Where was PaymentPreimage? I realized, the preimage is just an HMAC of some metadata, so they don’t have to store anything, yet are able to verify the preimage when a payment is received. Clever!

Started to review using Mastering Lightning. Got pretty distracted heh.

Creating a channel is composed of 6 messages: open_channel, accept_channel, funding_created, funding_signed, funding_locked, funding_locked.

LDK exposes a create_channel function that makes opening a channel easy. I was trying to find where it actually sends messages like open_channel, so I did some digging. When creating a channel, it actually just creates the channel state in channelmanager, then pushes a msg event to signal to the peer handler to send the message. The peer handler has a function poll_events (or something like that) that either the user or some nicely packaged language bindings will need to call, that spawns some thread that actually handles the events and sends bytes over the TCP socket. Events are everywhere in LDK.

Review: SegWit was important for lightning because txids need to be known in order to create commitment transactions to sign before opening the channel!

Did another cryptopals challenge, number 9. Struggled a lot for no good reason on challenge 10, think its an issue with openssl versioning. Going to try to switch to a different package or something next time I work on it. Ugh.

Not doing much today.

Did the last two challenges in cryptopals set 1.

Still feeling really intimidated doing things. I want to take action to get better, do better work.

Weaknesses of mine that stand out right now:

• Lack of understanding of what users of rust-lightning face

• Lack of familiarity with the nitty gritty details of the protocol

• Lack of familiarity with the repository

Setting out my plan to keep putting more lightning stuff in my brain:

• Read Mastering lightning (again) to get a good overview of everything.

  • Enhance my comprehension by looking at the bolts and rust-lightning implementation. This will hopefully help me become more familiar with the codebase as well as the technical details.

• Take some time before jumping into your next issue. You can contribute by doing lots of code review, and when you feel comfortable, and can understand an issue through and through, resume.

• In the meantime keep your rust skills sharp just working on the cryptopals challenges.

Completed cryptopals set 1 challenges 5 and 6. Challenge 6 was said to be a qualifying round, and now that I’ve gotten in the groove these didn’t take too long. Feeling good!

New helix tip: you can use C-d to scroll down and C-u to scroll up in the documentation popup!

Learned a new word: mantissa. It’s the numbers after the decimal point.

Stumbled upon this: Rust Performance Book.

Learned what inlining means: replace a function with the code itself to help performance. Mainly helps because the compiler can optimize the code together, as well as get rid of function call overhead. The Rust compiler already inlines a lot automatically, the #[inline] attribute however can suggest it to the compiler.

What are holding cells? Also seeing this all over but never really looked into it.

I can feel myself becoming a little looser when pushing to my PR now that I’m just responding to feedback more often. On one hand it’s good for me to become more comfortable, but also I don’t want to get sloppy and waste anyone’s time if I could’ve caught something with marginal extra care.

I’m going to continue doing more review, but dang it’s basically the feeling of not knowing where you are, just repeated over and over. I guess that’s the point; that’s how you get assimilated to a code base like this, keep exploring new places. Oh but it hurts my head. But this is only the beginning. Gotta do a lot more of this to do greater and greater work!

I’m reaching a point where I’m honestly sorta struggling with this. Need to just keep at it, keep doing code review, keep learning more about lightning, and importantly, keep having fun exploring. This is really cool stuff. If I don’t get overwhelmed with the minutia, it’s pretty crazy to think I get to work on this and have insane Bitcoiners spare their time to look at my code.

Got some more feedback on my PR, implemented it, marked it ready for review.

Learned git show and git diff. Difference between them, show is for single commits, diff is more generic and can do branches or any two commits. git show shows change for latest commit. git diff <target-branch>..<working-branch> is nice to see what your pull request is looking like.

Learned how to revert a merge.

Learned to rebase instead of merging main branch into working branch. In all honesty I can’t believe I didn’t know this before lol. Also realizing how nice rebasing is to just organize your different commits in your branch—will be doing this by default now.

What is a phantom payment? I see this all around the repository but I don’t actually know what it is.

Was reading some documentation about failure codes (PERM|15) where it talked about the fact that they returned differentiating errors in certain cases, it allowed for a forwarding node to be able to probe nodes with HTLCs of different amounts or cltv_expiry’s in order to find out the final destination. Pretty neat.

Started on cryptopals. Did the first 4 challenges of set 1. So far they were pretty simple, but I sometimes would get caught up overengineering or thinking it was harder than it was. Lacking GitHub Copilot here has been very annoying, and my Helix autocomplete always has somewhat of a delay. Will need to look into that…

Got a bunch of PRs to review after asking in the discord. Exciting!

Wrote blog post.

Merged main.

Had developer meeting.

Filled out summer of bitcoin application.

Interesting, to get a Channel from ChannelManager (with some htlc: ClaimableHTLC), you have to do something like this:

let short_channel_id = htlc.prev_hop.short_channel_id;
let (counterparty_node_id, chan_id) = self.short_to_chan_info.read().unwrap().get(&short_channel_id).cloned().unwrap();
let per_peer_state = self.per_peer_state.read().unwrap();
let peer_state = per_peer_state.get(counterparty_node_id).unwrap().lock().unwrap();
let chan = peer_state.channel_by_id.get(chan_id).unwrap();

• There might be a better way I just don’t know, but that’s how it seems to be done. I would’ve thought there might just be some hashmap of channel ids to channels as a field of ChannelManager, but it’s interesting it’s kept in peer_state.

Getting a lot better at Helix. Some nifty shortcuts: ]g to go to next change, space f for file picker, m i <char> to do vim’s v i <char> i.e. matching inside, space k to pull up documentation on a variable/type/etc., g d to go to definition, g r to go to references. Still having trouble navigating the jumplist, and going back to previous places/managing different places at one time.

Learned about the ChannelManager lock order. I ran into some trouble where I was violating the lock order, and my solution was to just clone short_to_chan_info to release the lock so I could then grab the lock for per_peer_state. It seemed there was no other way to find a channel from a short channel id, but it felt like cheating, although I saw it done somewhere else. I’ll hopefully come to understand more about why the lock order is the way it is as I work with the code base more.

Also noticing there are several different structs and enums defined relating to failure codes/messages that could maybe be standardized in the future.

Did a lot today. Finally feeling a little more familiar with at least the area of the repo I’ve been working with.

Worked on Illini Blockchain developer task.

Getting a little better at helix.

Seeing other people coming back to campus.

Difference between impl and dyn: they call it static vs. dynamic dispatch. Static dispatch is that the rust compiler will copy code for each type that fits the impl type. Dynamic dispatch is that it will store a fat pointer to both the object and vtable of functions, i.e. it has to deference two pointers to call a function at runtime.

• Most of the time wouldn’t you just want to use impl? Seems like adding extra code in compile time wouldn’t be that big of a deal.

  • Adding to this a week later: An example: let’s say you want to return different error types from a function. You couldn’t just do impl because you couldn’t statically dispatch, i.e. you couldn’t just make two different functions for each type because the returned type is determined at runtime.

• What about Box<dyn Error>? I see that used often. Why?

Helix. On one hand it’s sort of a distraction from the main work, but it’s also fun to learn something new. I think it could help while getting assimilated to rust-lightning if I knew how to navigate code easier. I could probably just get better at VS Code, but whatever.

Along the same lines of using a terminal-based editor, going to actually commit to customizing my terminal setup, iTerm, or at least try it.

Already realizing how much better iTerm is after just tweaking a few things. Also went through Helix tutor and it’s pretty interesting, and I get how a lot of things are intuitive like the guy in the video said.

Spent way too much time deciding between themes for helix lol.

Got some feedback on my PR! The maintainers are really generous with there time. It’s making getting started with the repo a great experience.

My intuition was right about the cltv_expiry wanting to be decreased for the HTLC in submarine swaps! I was pretty confused, but it makes sense now.

Struggling to test. Sort of annoying iteration is slow because I think it’s compiling every single test even when I am only running a single test. Sucks to not have my monitor right now either, my laptop screen is so small while navigating all this code.

Slowly in-taking more in-depth knowledge of lightning. Read part of BOLT 4 today for the failure messages section.

Reviewed rust-lightning contributing guide again as I’m getting ready to open that first PR on the repo.

This seems valuable: Learn Bitcoin from the command-line.

This also seems valuable: cryptopals. Could be good to do if I’m not writing much code at any point.

Jon Atack’s website has convinced me even further to try to do a lot more review as I onboard to open-source. LDK probably doesn’t have as much resource-constraint as Bitcoin Core when it comes to review, but it really seems like a great way to learn and contribute value at the same time.

Recurse. Seems cool.

Need to return to LDK meeting notes in the future to help direct my efforts towards what I might review.

Opened a draft PR, and made a comment asking about submarine swaps and using u16 vs u32 on a different PR. First step into doing more code review.

Discussion in LDK discord - certain aspects of LDK are very difficult to fuzz/proptest. Don’t exactly understand the details, but fuzzing seems to be an effort they want to improve. Also I don’t really get the difference between fuzzing and proptest.

Apparently a good book on computer networking suggested by the LDK discord

Finally learned about macros in Rust. They are honestly fairly straight forward, at least conceptually. This is my first real introduction to meta-programming, which I had heard a lot about but never really learned myself.

Also realized there are a lot of books for Rust stuff. There’s a reference book for more in-depth documentation of the language structure. Seems like a good section on macros.

Review: OP_CLTV (check lock time verify) is absolute, OP_CSV (check sequence verify) is relative to the confirmed block the transaction appears in.

Trying to dig into a PR: https://github.com/lightningdevkit/rust-lightning/pull/1878

• In the issue it mentions this is important for people doing (submarine) swaps: why?

  • Review: Submarine swaps. Can be used for…

    1. Alice wants to pay Bob on-chain, Bob wants to receive payment off-chain on lightning, so they use a third person (or some intermediary route) Claire that has the necessary liquidity to pay Bob on lightning. Here, Bob generates a secret, and sends the hash to Alice—essentially just Bob creating a Lightning invoice for Alice. Alice creates an HTLC to send money to Claire (Claire can take funds within time frame with preimage/secret, after cltv_expiry Alice can reclaim funds). Claire creates an HTLC to send money to Bob on their payment channel (only after Alice’s HTLC has been broadcasted on-chain to ensure she can’t double spend it). If Bob claims the payment, he shares the preimage/secret, and Claire is able to claim the funds from Alice.

    2. Alice is paying Bob on-chain to receive off-chain assets. Happens similarly to above but now both Alice and Bob are senders and receivers.

• What is all this about packing the custom_cltv_expiry_time into the payment secret metadata?

• What is this PR doing: changelog at the bottom was probably the most helpful for this.

Watched this video on Rust 2024. I didn’t even really consider that there is still so much development to do on Rust, and that some of the “accidental complexity” scenarios that were mentioned existed.

SCID stands for short channel ID.

Looking into this issue: https://github.com/lightningdevkit/rust-lightning/issues/1570

• This is kind of tough. I feel like it’s not that bad, but I just am not familiar with the whole repo so I don’t know what exists and where things are.

• I’m looking at pub fn fail_htlc_backwards(&self, payment_hash: &PaymentHash) on ChannelManager. This seems to be what the issue is talking about, i.e. an exposed function for the user to tell the ChannelManager to fail back a payment. From what I understand, this function removes the payment from the ChannelManager’s claimable_payments–the (inbound) payments it thinks are able to be/in the midst of being claimed, then for each source of the payment (can be multiple for a MPP) it “fails back” HTLCs to free up resources on channels along the path from here to the original sender.

  • Purpose of the function: Indicates that the preimage for payment_hash is unknown or the received amount is incorrect after a PaymentClaimable event, failing the HTLC back to its origin and freeing resources along the path (including in our own channel on which we received it).

  • For each source currently, it marks the reason as HTLCFailReason::reason(0x4000 | 15, htlc_msat_height_data); which I don’t really get, however I believe this might be something we would change to allow the user to input?

• Have a lot more to say, left my summarized thoughts as a comment.

Trying to go about my first issue is leaving me with a good amount of questions that I’m realizing could probably be answered by doing a lot of review. Definitely going to prioritize trying to do that tomorrow.

Visiting friends.

Noting another doc on lnurl: lud16. Kept seeing people use this term and didn’t know what it was.

Watched this podcast. Some takeaways/thoughts:

• Bitcoin doesn’t just help monetize on unused energy, it ultimately is pushing towards clean/renewable energy because it happens to be the cheapest, and so miners on clean energy will beat out the others, and capital/resources will accumulate towards clean energy infrastructure. Bitcoin in the long-term will only get cleaner, and bring in lots of capital towards clean energy infra.

• More on flaring: while extracting oil, natural gas is released, and so they actually burn it because it’s better for the environment than just releasing it into the atmosphere.

  • How does natural gas work? It needs to be used immediately?

• Blockstream was created as a way to let people do bitcoin full-time (because at the time everyone had full-time jobs and were hobbyists/enthusiasts)

• I forget that being a funded open-source contributor has only been around for maybe the past 5 years or so.

• Deep care is needed when people’s money is on the line, things move slowly.

• No technical reason to not decentralize mining/negotiating transaction sets within a pool, just less convenient; the protocol(s) needed to be invented.

• He mentioned he lately was working on some rust tooling for integrating lightning into existing wallets. Was cool to hear about the humble beginnings of LDK!

Surfed https://bitcoin.ninja/. Will return.

Noticed both jb55 and TheBlueMatt don’t use Github for their personal repositories. Decided to look into this. Didn’t realize there have actually been several instances of GitHub censorship. I wonder what more bitcoiners think about this, and what other solutions are for the sort of mass collaboration that happens on something like Bitcoin Core. Also, everything is trending towards me getting better at using VPSs in order to self-host things and just build more stuff.

Ran my own nostr relay locally to better test my greg bot. Tried to get it up on AWS EC2 but was sort of struggling. I am currently weak at configuring instances for exposing ports in particular ways and more importantly debugging errors in these situations.

Got the main implementation of my greg bot working. I can set a stream rule to only receive tweets from a user and run my bot to repost a tweet whenever it comes in. Next steps are to refactor, clean things up, and deploy it on some server so I can leave it running.

Not sure why you’d ever want to, but I realized you could actually have a function with a mutable variable of type immutable reference: fn function(mut x: &String)

Learned you cannot implement a trait on a type from another crate. You must make a wrapped type. I was trying to implement Clone on nostr_types::PrivateKey, which is a tuple struct of two clonable types, but I couldn’t. Had to make a wrapped type.

Finally learning asynchronous programming in Rust. For a while now I’ve been exposed to async in Javascript, but the few times I tried to read about it I never really understood it. I recognize how to use async/await syntax but didn’t understood what was going on under the hood. Now I’m finally reading more.

• Async functions are another tool for concurrent programming, like threads. However for situations where you have tons of smaller tasks, spinning up threads for each task takes memory/CPU overhead. Async functions allow you to execute more tasks than available threads at one time.
• In the chapter where they essentially build an async runtime (Futures + Executor), they have the TimerFuture spawn a thread when it’s created. I feel like this makes the example sort of confusing, because it seems they’re spinning up a thread for each task, which is seems like a situation where you wouldn’t need async programming. I think a better example would just have it make an HTTP request or something. Will revisit sometime. Otherwise, it was a really cool chapter!
• Basic summary of how an async runtime works: futures implement a poll() function that will return a result or that it’s pending. They’re giving a wake() function to call when they should be polled again. This wake function sends a message on a queue of tasks. An executor waits for tasks from this queue and continuously polls them to check if they are done. I probably fudged certain details—will come back to this.
• Extra review: difference between a thread and a process is that threads share memory space while processes don’t.

Taking a very brief intermission to work on what will likely be my life’s legacy: a bot to repost greg tweets to nostr.

Exploring nostr development through this project has given me some perspective on just how young this project is. There are a couple of rust packages, all have less than 30 stars on github. On nostr I see so many people talking about developer-related stuff, but I realized there are actually pretty few people actively working on it, even with the rise in popularity lately. It’s not that popular yet.

• Mainly using/copying nostr-types and nostr_rust.

Relearned that secp256k1 is the elliptic curve used in Bitcoin, and schnorr signatures in Bitcoin are a Schnorr scheme using this curve (as opposed to the prime/generators scheme taught in class). I should actually look into how elliptic curve cryptography works at some point. Learning the number theory for what we learned in class was super cool so ECC must be pretty interesting too. Hopefully I have enough math background to grasp it.

Choosing to use Rust for my bot because why not. Learned to make HTTP requests in Rust using Hyper, and can successfully make calls to the Twitter API.

Realizing I should just start working on rust-lightning instead of doing things on ldk-sample. ldk-sample really doesn’t need more work on it, and contributing to rust-lightning will help me immerse myself in the codebase better anyway. One PR was enough to practice, now I should just get started.

My first PR was merged!!!

Did a couple of programming problems. I figure I should probably practice a bit so I don’t bomb the technical screening for Summer of Bitcoin.

Learned what a transaction Output is:

pub struct OutPoint {
	/// The referenced transaction's txid.
	pub txid: Txid,
	/// The index of the referenced output in its transaction's vout.
	pub index: u16,
}

Neat glossary of hashes.

Looking into persisting payment info for ldk-sample-node

• There’s a Persist and Persister trait. Persist handles persisting ChannelMonitor which actually handles channel data. Persister handles persisting the ChannelManager, NetworkGraph, and Scorer.
• KVStorePersister allows you to implement one function and have auto-implementations for Persist and Persister. But this doesn’t get me anywhere for payment data.
  • This is implemented for FilesystemPersister which is a basic persister provided as a mini crate in rust-lightning.
  • Interesting to note the other things have default implementations for persisting but payment info doesn’t. Making note that this is likely state that is not needed to function as a protocol, i.e. past payments.
• The other thing, is to be able to persist anything, it needs to implement Writeable trait, which is an extension of the Writer trait, which is a simplified version of the standard std::io::Write trait!

Why do this issue at all?

• Saw it was an intended functionality from a TODO comment
• Seems like a relatively important part to demonstrate to a user. For sake of consistency and completion in the project, persisting this to disk seems logical.
• Could see how this is not super essential for this project because persisting this to disk is not vital to the nodes core functionality, and any added feature can increase complexity. But personally I think this would be good, and doesn’t add any more complexity than there already is.

It turns out we are actually on the third testnet. And this is reflected in the datadir being in testnet3.

Damus updates have been speedy lately.

Learned about LNURL and lightning addresses. Both are ways to have static text addresses (to acommodate BOLT 11) to make paying people easier, with LNURL being the underlying protocol. LNURL uses an address format: lightning:..., while lightning address uses the format: <name>@<domain.com>. The two address formats are just ways to receive metadata, LNURL sends the LNURL to the lightning service (or some sort of decoded data?) while lightning address sends to http://domain.com/.well-known/lnurl/username. They get back metadata, the user inputs and amount to pay, then it requests the wallet’s LN service or node to ask the payee’s node to generate a new invoice and tries to pay it. Pretty neat. I have mine from alby: alec@getalby.com and put it on my damus/nostr profile info.

Learned what SIM swapping is: Someone can pretend to be you using fairly basic/accessible information to get your phone carrier to reassign your phone number to a different SIM card. Then they can compromise anything that uses phone number for authentication.

Learned how to squash PRs/commits using git rebase -i HEAD~n from this resource.

Got some more feedback on my PR from before Christmas! Getting closer to getting it merged!

The other issue I was starting to work on got closed before I could share my update. Conveniently I was going to sort of recommend that anyway, so it all worked out. And my work on it does not go wasted, I definitely learned a good bit from spending time on it.

Got an answer on why not to use dependencies in a project. It’s interesting, when learning to code, you are taught that libraries save you from reinventing the wheel, and up until now I never really considered the downsides of adding dependencies. But it makes a lot of sense now giving it some thought, that for any serious project, a dependency is often a liability. It can take away from iteration speed, it can add complexity, it can add security vulnerabilities, and if you need to change something about it you might spend time maintaining a fork or wait to merge a PR for the next release. If you can reproduce the functionality simply, you face far fewer of these issues.

Visiting friends.

Did some New Year’s reflecting. Took a break after all the past week’s socializing.

Today I’m working on fleshing out some implementations of CLI history using the different packages.

• dialoguer seems sort of limited. It really easily handles adding history, but it takes away (doesn’t implement) certain features that are in stdin added by the terminal you’re using (option/command + backspace or left/right navigation) and it always outputs a “:” and you can’t change that default. It’s more for specific input prompts like selection prompts, etc than it is for just a CLI.

I’m realizing you mainly have 3 options for CLI UX:

1. Use Rust’s native std::io::stdin

2. To do anything beyond Rust’s native stdin, you must:

   • Implement all normal CLI features, e.g. . Someone can fact check me on this, but from what I could find, there isn’t really a way to just override or extend particular features of stdin, mainly because stdin only reads lines of input (using however the terminal—the one you’re already using the run the program—implements inputting text), as opposed to individual key presses or bytes.
     • This will likely require using a dependency like console unless you want to implement the different windows vs. unix stuff with a lot of unsafe rust + libc

3. Split your program into a daemon program + RPC CLI, and leave CLI features to be implemented by the terminal you’re using to run the program(s).

• My personal choice would be to leave it how it is, and if we wanted to add more advanced CLI features, should switch to daemon + RPC (as opposed to current interactive mode)

GPT is failing me. It’s great at generating ideas, or for giving you some code for something very common, but once you are asking for something that is relatively less common, it starts to actually waste your time. Both github copilot and ChatGPT have given me blatantly incorrect code/answers to some questions when asking about termion and console.

From what I’ve found, using std::io::stdin() you are only able to read and handle input by lines. For something like supporting up/down arrows to use a history of commands/inputs, you need to handle individual bytes and/or key presses.

To get access to a byte-by-byte input stream, there’s a crate called termion`, however it only works on Unix based systems.

There’s a crate dialoguer that helps build CLIs that handles a lot of stuff for you. However in lightning/bitcoin it seems people are generally reluctant to add dependencies. So if we want to just build the necessary pieces ourselves, dialoguer is built on console, which exposes the lower level interfaces. Any lower than that is using libc.

• I am kind of curious what is the mindset behind dependencies. It seems like some are added, but most of the time (from the little that I’ve seen) when the idea is brought up it’s face with a lot of reluctance. Is it mainly because it can be a security risk? Or that it can add unnecessary bloat or complexity to what is meant to be lightweight?

Default stdin doesn’t handle much besides typing, backspace (including option + command/ctrl), and enter. So any arrow keys do not work.

Learned about Mac stage manager. Pretty nice.

Went through these slides on Lightning as a more secure communications layer distinct from the web. I need to look into the lightning part more (BOLT 08 + 10). From what I understand, the big problem with the web is TLS. HTTPS relies on people trusting certificate authorities (CA), which can be compromised, particularly intermediary certificate authorities, e.g. actually happened with the Iranian government snooping on Gmail traffic. To establish an encrypted connection, the client first verifies the public key of a server by checking if it has a digital certificate from a trusted CA (on TLS). However CAs actually delegate to intermediary CAs which our browser also just trusts I guess? Intermediary CAs (and technically root CAs) can be compromised, i.e. inauthentic certificates can be made, and now we’re sending messages to someone that isn’t who they say they are.

• I’m honestly not sure how communicating over lightning solves this, as with any communication with someone over the web for the first time without prior contact, won’t you need some level a trust? I think Lightning may be more secure because you can directly communicate between nodes instead of having to host stuff on a separate web server that you have to trust?

• Another thing: DH is forward secret while RSA is not. Forward secrecy means if an attacker finds out a long-term secret in the future, they cannot decrypt previous conversations. Anything that’s not like DH is pretty much not forward secret, as a private key is used to decrypt messages, so if it’s revealed then the attacker can decrypt. DH is forward secret because even if an attacker discovers one person’s secret, they don’t necessarily have the shared key. The only way would be for the attacker to also record the initial handshake in order to obtain the other necessary information to find the shared key. (asked GPT)

Learned about Valera Labs through this podcast episode that I saw posted in the LDK discord. They are building lightning infrastructure, mainly back-end services that enable easier non-custodial lightning usage. Sounds like they’re doing a lot of things. One is indra, which is a “node runtime,” which is basically software that manages many different nodes at once, solving the “always online” problem by spinning them up in an instant whenever needed. Uses LDK for custom nodes. In the talk Azz also mentioned they have a great LSP going, as well as how their whole architecture is meant to operate very well with cold wallets, similar to VLS. Also a large emphasis on custom hardware. Note: this tweet talking about indra progress. The main guy Azz is 16 years old, and the company is based in Cayman Islands. Seems pretty hardcore. Another cool resource on Indra.

• One thing I will note is although it’s technically non-custodial, they are right to say it’s “semi-trusted.” You own your own keys, but all of the channel state and node data is on their servers. Nevertheless, this is a huge value add, and serves its own purpose in the ecosystem. I think for many users, what Valera is building will probably end up being their preferred sweet spot between self-custody and ease of use.

Zero-conf channels. Basically exactly what they sound like. Funding transaction doesn’t need to be included in a block yet to start using lightning. Only fit for use with trusted peers. Can enhance UX with trusted LSP/wallet provider. Was added to rust-lightning in June 2022.

Rapid Gossip Sync. A big performance bottleneck is that when a node comes back online, it needs to get up to date on the state of it’s channels, and also other public channels in order to do routing. Up until now, the basic Gossip protocol to communicate this information is assumed to deal with nodes that are online, that are continuously listening for channel updates and announcements. Given this, they include a lot of redundant information. If you are trying to run a node on a mobile device, it might take up to a couple minutes to get synced, which hurts UX if you just want to open the app and send a payment. With rapid gossip sync, you can deploy a server to take snapshots on some regular interval, and it can significantly reduce (50MB → 5MB! ~0.4 sec sync!) bandwidth required to sync because it can omit a lot of data knowing your node is receiving this big batch. This can make it more practical to route privately on a mobile device without giving up your privacy to third-party servers or sacrificing UX.

Found new issue to do that will be good for helping me learn about other parts of the repo: persisting payments data to disk

Commented on new issue and started looking into it: CLI history/up-down keys

Got a weighted blanket and silverware for Christmas! Win!

Watched this video on composition vs. inheritance. Seems it usually makes your code more adaptable when you use interfaces over abstract parent classes, as it’s easy to change a class to implement or not implement an interface, but if you want to change something about an abstract class, you likely need to change many things about your child classes. The downside of interfaces being if you have a lot of boilerplate, a parent class would be nice there. Not sure about many other languages, but I know in Rust you can implement default behavior for traits which seems kinda like a best of both worlds.

• Really liking this channel! This is the kind of style of educational videos I would like to make about lightning concepts eventually. Thinking the basics of what he’s doing are reproducible in a software like keynote.

git commit --fixup <commit-hash>. A maintainer left some feedback mentioning they usually do “fixup” commits while iterating on review. Didn’t know this was a thing but that’s cool and more versatile/looks better than force pushes.

Started using Warp terminal a bit. Sorta inconvenient it’s not right inside my VS Code, but there’s maybe a way to get the features inside there..? Nice to have the better visualized autocomplete. Otherwise need to explore more.

Got a bit of feedback on my PR and responded to it. Feels good to have some more back and forth, and I think it will probably get merged sometime after Christmas and whatnot.

Figured out a way to test on Windows today. Seems that there is no way to easily run windows on a mac, which I guess makes sense. But there is an easy way to run linux on both mac and windows, that being docker. This seemed like too much work, so I spun up a Windows EC2 instance on AWS which makes you connect using remote desktop client which was super slow. However, I did feel pretty empowered by the fact I was able to do something I’d never done before pretty quickly/easily.

Learned about cargo clean. cargo build creates the target folder with build artifacts, but these can get corrupted/outdated and cause build problems. cargo clean just removes this directory from what I can tell which forces cargo to rebuild from scratch which can solve this build issues. ChatGPT helped me discover this when I was having build problems on Rust 1.48.0 but no compilation errors.

Did quite a bit of refactoring today in my PR I’ve been working on. Feeling pretty good about my changes now. Feel like the code is pretty readable. Main thing I could see feedback being on is just the practicality of this whole PR, i.e. is it really necessary to add this functionality to what is meant to be a relatively simple sample node; is it adding too much complexity? We’ll see. Got a maintainer to approve running CI and it passed. Hopefully will get some feedback tomorrow morning and I can get this merged sometime in the near future. Getting closer to my first contribution!

Becoming more and more familiar with all the ways to unwrap and handle Result and Option. Check out this block:

get_rpc_auth_from_cookie(None, Some(network), None)
    .or(get_rpc_auth_from_env_file(None))
    .or(get_rpc_auth_from_env_vars())
    .or({
        println!("ERROR: unable to get bitcoind RPC username and password");
        print_rpc_auth_help();
        Err(())
    })?

Pretty nice right?

I’m starting to understand more about what there is to work on in lightning. One aspect is infrastructure.

• The lightning network is strongly analogous to the internet. It’s a network of channels and nodes, but instead of sending information packets, you’re sending sats. And instead of just transporting data for free, you’re moving channel volume, this makes things harder. And instead of just making everything basically public, you’re trying to supply tooling to allow people to protect their privacy.

• Recently learned about LSPs. Just like the internet, the network doesn’t just work if people have computers. There’s a lot of infrastructure that will go into making the lightning network work as seamlessly as the internet does today. LSPs are one part of that. They allow for immediate inbound capacity, rebalancing channels, routing, opening channels, and more. And in a similar way that connecting to a pool of nostr relays allows for properties of decentralization is one misbehaves, having competing LSPs can have a similar effect.

Why can’t you just give poor people money? It’s a temporary patch on a systemic wound. You can give them money, and they may help people temporarily, and it would seem like you’ve essentially taxed the rich and helped the poor. But it causes inflation without actually empowering poorer people to earn more, so after some years it’ll be back to a similar situation. Maybe if you gave them a lot of wealth, or distributed it greater over time or something, but that probably requires more money and execution than is reasonably feasible. It makes sense that people try to target other forms of economic empowerment, i.e. increasing access to opportunities to build wealth, whether that means access to financial tools to be able to save, invest, or start a business more effectively, or education to be able to gain the skills required to work a higher-paying occupation.

Heard about Sensei in the past week looking into LDK stuff, but just discovered John Cantrell. This thread was great.

Lately finally concretely got the difference between source-based and destination-based (forgot the name, IP based?) routing: source-based is that the sender finds the full route (allows for onions → privacy), dest-based is where you can reveal yourself and the destination but now each node along the way can help out in determining the route. Pretty simple concept but for some reason never totally clicked. Interesting, a VPN is basically a trampoline node.

No println!("{var_name}") until Rust 1.57.0.

Setup my NIP-05 identifier.

Responded to some feedback on my PR and added functionality for finding and parsing the .cookie RPC auth file. Did some pretty solid refactoring, and in general my code seems quite simple. Yet it is still somewhat surprising all the different cases that come up for simply getting some credentials.

Continuing on my Rust journey. Learning all the different functions for handling Results and Options such as or_else, ok_or, map_err, and more.

Also learned about some Rust filesystem stuff. Can’t specify ~ character in a path which I guess makes sense since that’s an OS specific thing. And for finding $HOME, cargo and std apparently use a dependency, and have a deprecated std::env::home_dir function. Something funky that I wouldn’t expect from Rust. Honestly realizing Rust has a fair amount of clutter/things that are not super clean, but it’s still good. It’s just also like every other programming language in this respect. Nothing’s perfect.

Thought about the practicality of multi-path payments. I thought I heard somewhere that MPP are a cool concept but not actually that practical. I was thinking about it and it does make sense. In theory it seems like this payment channel system can unlock this ability to utilize liquidity in many place to route larger volumes. In practice, not only are you scaling up the amount of routing compute you have to do (lots of probing and knowledge of the network required to find multi-path route), the likelihood that all these paths exist and work may be very low. I can imagine how someone might make it atomic so I’m not worried about that at least, although I would like to look into that a bit more. My mind’s not made up however, still think it could have potential. Could at the least be very useful for just having more routing options across smaller number of hops.

Learned about rpcauth and .cookie from this and this. I can understand the difference between the deprecated rpcuser and rpcauth fields in the bitcoin.conf compared to the new rpcauth: The previous version was storing the plaintext credentials in the config file whereas the new field stores a salted version of the password. This is more secure because it doesn’t explicitly reveal the password if somehow the contents of the config file were leaked (but the attacker doesn’t have access to the local filesystem otherwise..?). My understanding of .cookie is that it’s a default rpcauth for whoever has access to the local filesystem already. So Bitcoin Core requires some sort of rpcauth at all times, and the cookie is a convenient way to grant/use it for the user who controls the node. Also contents of .cookie file are <rpc-username>:<rpc-password>

Opened my first PR on the ldk-sample-node repo! Will probably ask for feedback in the discord tomorrow! Ran into a couple interesting bits: 1. for doing multiple matches in an if let instance you must put your values in a tuple, 2. setting a variable equal to the result of an if or match statement is not as versatile as I once thought, but for good reason, mainly you can only use the non-return notation for returning values so you can’t return early unless you want to return from the entire function. Interesting to compare this to closures as parameters.

Refreshed my self on testing in Rust. Testing private functions is always done in the same module. If you want to run some setup before tests, you can only do that in integration tests, which are found in the folder /tests/.

Learned if you git commit --amend after pushing to a remote repository, you don’t have to clutter up your commit logs by pulling. Instead just force push: git push --force.

Learned more about nostr! Tried setting up my own nostr-rs-relay, but it was too late at night so I stopped part way. Listened to this conversation and felt their excitement. Great way to describe protocols is connecting to the idea of email. Also in describing nostr specifically: anyone can run a relay, you don’t have to run a relay to use it, we get all these decentralization properties by just connecting to relay pools and sharing relays.

Spent the day traveling.

Spent some time exploring nostr.

Watched this video. Didn’t really learn anything in particular, but got some broad exposure to how lightning is integrated into enterprise software.

Watched this video. Learned Bitcoin core has this concept of no free relay. Whenever you are replacing other transactions with a new transaction, you have to pay 1 satoshi/vByte that you are replacing in fees. If you didn’t have to do this, you could DDoS the network by repeatedly sending transactions and replacing them with negligibly higher fees and forcing everyone to waste tons of CPU bandwidth. This makes things tricky with anchor outputs because your counterparty can sneak transactions into the mempool that are unnecessarily large to force you to pay more.

Read this article. Seems kinda just like normal onion routing but with rendezvous routing? Refreshed myself on how normal onion routing works from this article: source-based routing, encrypts each step along the way, then sends whole packet to be decrypted one by one, uses special scheme to obfuscate packet size/position in path. What’s the difference between blinded routes and rendezvous?

Refreshed myself on vByte/weight-unit terminology. vByte = 4 weight units. Max block size = 1 MvByte = 1,000,000 vBytes = 4,000,000 weight units. Legacy (non-segwit) transactions byte and vByte sizes were the same. Segwit transactions allowed for parts of the transaction (mainly the witnesses) to have a byte = 1 weight unit instead of 4, which is how the block size was covertly increased in Segwit.

• Note: I previously thought the max block size is 2MB, but that’s actually just generally an upper limit for average block size. The actual max block size is 4,000,000 weight units, which can be up to ~4MB, but only if transactions are formatted super weirdly.
• Another side note: kB = 1024B = 1000B.

Learned a bit about Signet. Another test network that is meant to be “predictably unreliable,” and it requires an extra signature commitment something witness for a valid block. Will look into this more some other time.

[pub(crate) notation](https://doc.rust-lang.org/reference/visibility-and-privacy.html). Can make something

Crates and modules in Rust overview. More in the textbook.

Surfed the issues on ldk-sample-node repo. Might be an even better place to start than rust-lightning good first issues.

I tested out the LDK sample node and played around with it interacting with polar.

I watched this walkthrough of the main code for the sample node on bitcoin developers youtube channel.

I learned a bit about how LDK was used to bring lightning to cash app.

Learned how C++ smart pointers (unique_ptr, weak_ptr, shared_ptr) helps reduce mistakes in freeing up memory (memory leaks) and can mimick aspects of Rust’s borrowing/ownership system, however Rust’s compiler does much more than just that. Importantly, other features of Rust and it’s compiler enforce correctness/well-written code at compile time whereas implicitly a language like C++ enforces correctness at runtime by crashing or breaking. A prime example is the Option type, where if a variable may be None/null you are forced to check it as opposed to the possibility of it segfaulting in C++. Article

Surfed the good first issues on the rust-lightning repository and got a general breadth of where I might be able to start soon.