On 3/12/2013 9:35 PM, ZPO wrote:
I view the NW-MESH and HamWAN projects as complimentary. HamWAN in
general and the PSDR component in particular makes an excellent
backbone transport network to move data around the wider area.
NW-MESH works well to cover the last mile.
This is where I think HamWAN is being sold short, since it also works great
on the last mile. Reason being is HamWAN sites are all planned to be
non-ground-level, so LoS probability increases which makes last mile happy.
Your trip may be physically longer by a few miles, but the latency and speed
will still be good, if not better when compared to multiple ground hops for
the same distance. In close-proximity situations, the end-user-site to
end-user-site direct connectivity of NW-MESH I believe provides an
optimization. A COUPLE hops of MeSh might indeed be superior to traversing
a busy sector.
I don't think I'm selling HamWAN short, just looking at the likely
timelines to have those cell-type sectors densely deployed and seeing
it stretch into years if ever. It is good to have multiple things in
the network gene pool. Let the use cases shake themselves out. My
thought process is that a few clubs with advantaged sites can form the
initial connections from MESHland to the PSDR. I'm being brief, this
makes a lot more sense with a whiteboard.
That's a common misconception, that the cell sites need to be
densely populated. We're not like the cellular telcos you are
familiar with who have to cater to tiny antennas in people's pockets
that run barely any power, and shoot through buildings. We're
targeting fixed installs which can provide 30dBi gain and run 1W of
power, via LoS links. The projected coverage radius of a cell site
in point-to-multipoint mode is 25 miles. Just a handful of those
cover much of the Puget Sound. We're not talking years. Improving
coverage would be an ongoing effort, sure, but to initially blanket
the whole Puget Sound is not years.
Maybe we can grab a cold beer at SEAPAC and I'll tell you the joys of
trying to tie together US and NATO networks in Astan. All kinds of
messiness.
Sounds like a good time.
Breaking between these two paragraphs as this is one of the first
spots where the ethos of HamWAN and the MESH programs (the one here
and others) tends to differ. There is no "constitution" in MESHland.
That is by design. There are sets of interoperability recommendations
that can be thought of as barebones versions of the RFCs underpinning
the global Internet. I don't particularly care what hardware somebody
brings to the party as long as it has the right interfaces and can
speak the right protocols. This allows the network to grow
organically with each node extending the network a bit more. It also
tends to make things very resilient.
Yeah, all the constitution/organizational business in HamWAN land is
so that we can channel money in such a way that noone gets
offended. It would be good if HamWAN were hardware independent, but
WiMAX/LTE gear is just too expensive to make this a reality. So
we've taken a compromise on the hardware front. Technically HamWAN
doesn't care what hardware you use either as long as you speak the
protocols, but the protocols happen to be offered by one vendor
(right now).
More deeply though, I believe it is the lack of centralized funding
that keeps projects like SeattleWireless from taking off. The
investment is made either on the ground by individuals buying nodes,
which have a high attenuation path between each other, or by a
handful of enlightened individuals who seek height, but for a lack
of structured broad funding, run out of personal money to really
build and sustain something large. HamWAN pools the financial
resources of the larger community and invests where the biggest bang
for the buck exists: at high, strategic, locations. Such a communal
investment naturally gives rise to expectations (on part of the
donors) to have it protected, and therefore the need for the HamWAN
organization which maintains the investment and keeps it in good
health.
The raw MESH approach does have theoretical merit, but the window of
optimal behavior is quite small. Too low a user density, and you
can't see each other to MESH. Too high a user density, and you've
got a slow or unusable network. There exists a small window of just
the right user density that allows a MESH network to work properly,
but the dependency on such a small window is what gives me the
impression of it being a fragile design.
In the HamWAN design, a low user density is best served by sparse,
tall sites. Increased density (which comes with increased funding)
is met with more numerous site deployments, each having a smaller
coverage area. This model scales continually as the network evolves
in either direction (bigger or smaller) through time.
In terms of the resiliency, what's stopping a NW-MESH node from
announcing a whole bunch of nonsense routes and hijacking traffic?
This is where I think you, Rob, and the rest of the NW-MESH team need to get
together with the HamWAN people to spell out exactly what it is that each of
us is trying to accomplish. Do you have a mission statement you can share?
Do you have a favorite low-noise public meeting location with power + wifi?
Starbucks is sounding likely.
I'd like to see some more detail
from Bart on what he sees as the large issues with bidirectional
interconnection (I think I know most of them) or simply using the PSDR
as transport via tunneling between access nodes.
So even though it's possible for HamWAN to act as a dumb pipe for NW-MESH,
this is not a fair arrangement for HamWAN. If this were a peering agreement
at an Internet exchange, it would be considered a violation of the terms of
service set out in every Internet exchange I'm aware of. A fair peering
agreement is where each network moves equal amounts of data to and from a
peer, where said traffic is between clients of each network. This means not
via a peer network, and the traffic is from/to clients of the same
originating network.
This concept probably needs to be fleshed out a bit. I didn't
adequately define it in my initial post. Better with a dry erase
board.. My goal would be to have every node in MESHland offering
services to the wider network do so via a 44-net address which will be
advertised as an HNA4 on the network. This is easy to link into
HamWAN at interconnection points via BGP. That completely isolates
the internal 10.x.y.z/8 2.4GHz addressing (which isn't a viable
long-term solution for lots of reasons) from HamWAN.
OK, but this doesn't allow for the MESH nodes to talk to each other
via HamWAN should their ground links fail, right? Seems kinda
limited in utility. Will these 44-net addresses be advertised to
the Internet? Individuals getting allocations of 1-16 IPs directly
from AMPR won't be allowed to announce. You need to aggregate
somehow up to at least a /24 to qualify. This in most cases implies
suballocating from an upstream carrier who is announcing for you.
That's the HamWAN plan for its users. What's the plan for NW-MESH
users?
What IS the viable long-term solution for the NW-MESH internal
addressing if 10/8 is not it?
This is a good
thing. On the other hand, there are some functions and features which
add value to MESHland that require direct communication with OLSR ETX
values intact, and routing staying on 10.0.0.0/8. Some folks may
choose to not go with 44-net addresses and advertise their services to
the wider world. Someone traveling out of area may want their MESH
devices to transparently link up with other networks.
For the HamWAN mobility case, I'd like to have other networks with
their own 44-blocks all dialed into a VPN to support the exchange of
mobile traffic. That means if a user leaves their HamWANOregon
network and carries a sub-allocation of his 44.x.y.z/28 or whatever,
and then joins the HamWANWashington network, his personal subnet is
still fully routable on the Internet and via RF by way of
HamWANOregon and HamWANWashington exchanging that route via VPN,
even though the /28 announcement would be too small for the Internet
and would probably violate BGP route filters with our respective
ISPs. This way noone needs to compromise. Permanent re-location of
networks might cause some excessive traffic though, so re-addressing
of the user's subnet might have to be done. But that's where DNS
makes things nice.
The statement " If this were a peering agreement at an Internet
exchange, it would be considered a violation of the terms of service
set out in every Internet exchange I'm aware of" reflects a commercial
service provider mindset. This is not such an environment.
Well, the commercial rules arose out of general principles of
fairness. I'm not suggesting money change hands to offset traffic
imbalances, but I would like to make the traffic exchange fair.
What "fair" means exactly is a complicated thing, and we need to
hold a few discussions about it I'm sure.
If the
ham community as a whole contributes to fund, build, and support
HamWAN then HamWAN needs to accept uses that may not fit its model.
In reality, HamWAN donors will be driving the creation of the
network. It is to them that I feel primarily beholden, not the
general ham community. For example, if sector users are finding
their speeds going to zero due to inter-MESH traffic, HamWAN will
likely react to make the service work for the people who made it
happen in the first place. I consider this a fair approach,
although it is certainly not an everyone-is-equal approach. You
don't face this problem in NW-MESH since you're not centralizing
your funding, so everyone can be equal. But on the flip side of the
NW-MESH approach, any joker can come online and deny the network for
others, no? How do you deal with that kind of situation? In the
HamWAN case once abusive behavior is identified, the offending
client can be booted from the network.
I
don't expect there will be huge amounts of traffic flowing via
tunneled connections, but it is an important feature.
I'd like to see detailed models on this. What sticks in my memory
is the Auburn meeting I attended, and just the one room of routers
(20-30 of them maybe?) produced so much traffic that you could
barely get a ping through the MESH there. I expect they were all
running 54Mbit to boot, due to their close proximity. Sadly noone
was able to do an over-the-air protocol analysis to see what was
going on.
What I don't
want to see happen is money, site availability, and goodwill expended
needlessly operating two parallel networks. There will be compromise
and adaptation on both sides.
Yup!
The most important thing in such an interconnection is harmonizing
DSCP marking for QoS. If we want to get very granular and complex,
DISA UCR 2013 (
http://www.disa.mil/Services/Network-Services/UCCO/UCR-2013-Draft )
provides a detailed setup (section6 -
http://www.disa.mil/Services/Network-Services/UCCO/~/media/Files/DISA/Services/UCCO/UCR2013-Draft/06UCR2013Section6.pdf
). In the long run, as long as both domains use the same QoS, things
are manageable. I've played the game where DSCP tags are remarked on
traffic ingress/egress. It isn't fun and generally ends up breaking.
Yup! And QoS is an open engineering problem on the HamWAN side
right now. I don't think the IPv4 DS field will be the answer
either. We may actually be looking at rolling VLANs to solve this
without compromise. The problem is we actually want to extend QoS
down to the clients' internal networks so that your VoIP phone for
example will have a higher QoS than your PC. At the same time, it'd
be handy for the same PC to normally ride a "normal" QoS tier, but
when you know you're gonna move 100GB of data and you're not in a
hurry, that same PC ought to declare to the network "hey, use a
best-effort QoS". This may not be implementable given current
technology, but we'll shoot for the stars and hopefully reach the
moon.
Have not seen that mil spec. Looks daunting. :)
--snipping the BCWARN stuff since it isn't germane --
*gasp!*
In most cases, the difference between Part97 and Part15 is a state
of
mind. It is only when you start using frequencies outside the
ISM/UNII bands or running higher EIRPs on point-to-multipoint
links
that things become definitely Part97. For example, a 5.8GHz PtP
link
between two sites running 500mW radios and 26dBi antennas could be
considered Part15 or Part97.
This is not the case in 5GHz land. EIRP is limited to 1W under
U-NII. Anything beyond that and you gotta claim part 97 (or
possibly ISM?). I have not looked at 2.4GHz, but you may want to.
I suspect the rules will be similar. See our Spectrum
Allocation page for a detailed to-the-point analysis of the
official publications with sources cited. Apologies in advance for
the wide format.
There may be value in considering it
Part15 as we may want to run the link with encryption or perhaps run
an encrypted tunnel for a served agency through the link. In such an
arrangement, we might get the served agency to fund the link, we
install/maintain it, and they get a portion of the available bandwidth
as a backup communications channel. That isn't a bad deal for any of
the involved parties. Such a setup is being mulled now for one
group/area in particular.
Probably the best idea is to continue discussions at an exploratory
level and then spend some time with a couple dry-erase boards at
SEAPAC. I will be there Friday-Monday (driving back Monday). We can
enjoy some frosty beverages and dry-erase our way through most of the
challenges.
I agree on the white boarding, but the timeline you propose is
really far away. It's going to become harder to make any necessary
changes on HamWAN/NW-MESH as the networks grow over the next few
months. I'd like to see us reach some agreements well before
SeaPac.
--Bart