Monitoring Infrastructure Deployment (ispmon) BOF IETF 57, Vienna, July 17,
2003 Meeting Minutes
Reported by Gianluca Iannaccone (chair) based on notes from Matthew
Zekauskas.
Agenda bashing
1. Presentations: . Deployment of inter-operable and
cost-effective monitoring infrastructures in ISP networks (Gianluca
Iannaccone, 15') . Improving measurement and monitoring for ISPs (Nevil
Brownlee, 10’) . Sprint’s Continuous Monitoring project (Gianluca
Iannaccone, 10’) . IPPM Applicability Statement (Henk Uijterwaal,
10’) . Multi-domain monitoring across European Research Network
(Nicolas Simar, 10’)
2. Scope and candidate charter (15’)
No comments were made on the agenda.
-- Gianluca Iannaccone presented the proposal draft
"draft-bhattac
haryya-monitoring-deployment-00".
Three main application areas for monitoring: i) network resource usage, ii)
traffic accounting, iii) fault diagnosis. The three areas have
different requirements in terms of timescale of interest,
granularity of information and location of measurement devices.
The first area has been addressed extensively (OPSTAT, SNMP, RMON). The
second is currently the focus of IPFIX & PSAMP working group. The third is
partially addressed by IPPM. Some relevant aspects of monitoring (namely
storage, aging of data and control plane functionalities) are not
addressed by any working group.
. It was observed that traffic accounting is also present in the
charter of the TEWG but that is has not been mentioned in the draft or in
the presentation.
. Matt Mathis observed that a lot of the aspects touched in the draft have
already been worked on at the IETF. Previous efforts have failed to
converge to a solution because they are really hard problems. For
example, the OPSTAT WG addressed the problem of a common format for
statistics and ended up arguing for long times about 5 minutes
averaging intervals. The same has been experienced in the IPPM WG. The
idea was to have an algebra of metrics to be composed, i.e. divide a long
path in sections and be able to characterize each section separately and to
compose the results to understand the performance on the entire path. This
effort has also not converged.
A second concern is that some other issues mentioned in the draft seem to
belong to the conversations between ISPs and monitoring equipment
vendors. Solutions in that area permit vendor differentiation. The IETF
does not belong in that space.
. Jon Bennett suggested that the IP Measurement Protocol (IPMP) could be a
viable solution for the metric composition problem but that it does not
have a WG where it can be discussed. Matt Mathis agreed with Jon
Bennett but objects that a group more focused on a specific problem would be
able to converge to a solution.
. Gianluca Iannaccone argued on the second concern (ISP/vendor issues not of
interest for IETF) that an ISP would like to standardize the way data is
collected and stored not the specific analysis or statistical methods
applied to the data. That area belongs to the IETF and there is
interest from ISPs for two reasons: i) ISPs want to have access to
multiple vendors, and ii) ISPs want to be able to share part of the
collected data.
Matt Mathis opposed that sharing statistics between ISPs is something that
comes from university world. It is unlikely that ISPs will be willing to
share data ("give away maps of wallets").
Gianluca Iannaccone claimed that for network resource usage & traffic
accounting there is not much to be shared but for other issues (e.g.,
fault diagnosis or incident reporting) sharing would be very useful.
. Rudiger Deib asked what IPFIX, PSAMP are doing wrong that justifies the
need for a new WG. Gianluca Iannaccone answered that they are focused on
specific applications (traffic accounting area) and do not address some
practical problems ISPs currently have (such as storage, aging, control
plane).
-- Nevil Brownlee presented a slide show about "Improving
measurement and monitoring for ISPs"
ISPs have always monitored their networks. Recently, they also started to
give near real-time reports of network performance via web pages.
However, measurement is still an art underpinned by science. An ISPMON WG
could improve this situation, thereby helping ISPs. Need to find better
ways to "use" network data. Collection is always easy. Storing,
visualizing and analyzing data is difficult to do. There are tools (MRTG,
Argus, snort, bro, ...) but can we do better? Would it be worth looking
again at something like the OPSTAT reporting scheme (RFC1857)?
. Nevil Browlee agreed with Matt Mathis that these issues are hard
problems addressed in the past and that did not get too far. There are a
lot of different measurement infrastructures out there but they work only
with fairly well specified goals. To build a general
infrastructure is really hard. This BOF should define a reasonable
number of problems where we can make progress.
. Nick Duffield asked how the metrics presented in the slides (slides 3/4)
sit with activities in IPPM WG.
. Naharito Hakuda (?) declared that the slides are of interest and asked
the speaker what he thinks about itrace technology. Nevil Brownlee
answered that the traceback of packets across providers is an active
research area. It is an area that providers should be interested in but
currently have not enough experience to say which way is the right way.
. It was observed that the example shown in the slideshow appears to refer to
a work addressed in existing WG, IPFIX and IPPM. Gianluca Iannaccone
answered that this BOF would not standardize things addressed in other WG
but there are two aspects not addressed: i) feedback from ISPs on what
applications are most interesting and ii) what to do once the data is
collected (storage, aging) is not defined there.
. Question from the audience: "Why not standardize also those aspects in
those groups?" Nevil Brownlee answered that no single WG covers all
metrics. So one useful thing ISPMON could do is to create a laundry list. It
could be a group for sharing experiences.
. Merike Kaeo, co-chair of IPPM, mentioned that in IPPM and in TEWG there
are similar efforts going on. IPPM has not been able to find anyone in ISP
community to contribute to the WG. Also the TEWG has a document on
requirements for measurement where chairs are trying to get input from
ISPs. It seems that this is exactly what ISPMON is trying to do.
-- Gianluca Iannaccone presented a slide show on Sprint's effort in
monitoring the IP backbone network. That effort is the reason behind
proposing the BOF and the draft on monitoring infrastructure
deployment.
No specific comments.
-- Nicolas Simar presented a slide show on "Multi-domain monitoring
across European Research Network". Five different research network have
built a team to exchange measurement data across domains and share the
monitoring infrastructure among providers. See slides for more details.
. Rudiger Geib asked if the effort is on concatenating measurement done by
different tools or also end-to-end measurement as well. Nicolas Simar
answered that end-to-end is out of scope here. However, if end-to-end is
available it could be used to compare it with concatenation of
measurements to see where a problem is.
. Rudiger Geib asked then if in case of different tools you share raw data
or evaluated data. In case of evaluated that is not addressed in IPFIX or
PSAMP. But then it is out of scope for ISPMON as well. In case of raw data
NOC would have to be aware of every data format. Nicolas Simar
suggested that raw data would be better and that providers need to agree on
common format per metric.
-- Henk Uijterwaal presented a slide show about the IPPM
Applicability Statement. There are many open parameters in the metrics
defined in IPPM. Not simple to configure. Feedback from operators would be
helpful to identify what to measure, when to measure and how to
configure. So far, it has not been successful.
. Bert Wijnen observed that if operators do not give feedback then there is
no reason to do it. Henk Uijterwaal answered that operators always said
they are interested. Merike Kaeo noted that privately all operators think it
is a good idea but noone would provide input to the mailing list.
Rudiger Geib observed that may be it is because there are very few
one-way delay systems installed.
. Gianluca Iannaccone pointed out that one of the problems providers have is
that out-of-context questions on applicability are hard to answer. The goal
of this BOF would be to provide that context.
. Dave McDyson confirmed that within MCI there is interest in IPPM but
difficult to attend WG there are higher priorities. It is not clear
whether a new WG would help at all. It would be valuable instead to have
service providers to meet informally in a forum. It is not clear that this
is a valid justification for a WG.
. Randy Bush (AD that gave the OK to this BOF) objected that there is
still no driving reason for a WG. None of the issues pointed out by the
chair appear to have traction. Underlying the discussion there is the
standardization of disk formats and data exchange rather than what
measurement metrics are of interest.
-- Gianluca Iannaccone presented to the room the draft charter for the BOF:
1. Provide BCP documents . what ISPs need to monitor and what metrics. .
how to instrument monitoring systems in large-scale provider networks. .
describe known-to-work implementations and open issues. 2. Specify ways for
ISPs to share/compare monitoring data. . common metrics and analysis
methods. . common formats for collected monitoring data (e.g., packet
traces) 3. Specify components of monitoring infrastructure not
addressed in existing WGs . storage/aging of collected data .
statistical analysis of traffic . control plane functionalities 4. Make
recommendations to other WGs standardizing different elements of
monitoring infrastructure . e.g., IPFIX, IPPM, PSAMP, INCH, IDWG, TEWG,
etc.,
. Randy Bush asked for the projector to show the table of content of RFC
1857 where one chapter is dedicated to storage format. That RFC is dated
1995. Gianluca Iannaccone rejected the idea that RFC 1857 is
applicable to today's measurement infrastructure. RFC 1857 focus was only on
network resource usage metrics (utilization, etc.) not on passive
measurements, flow measurements, etc.
-- The chair closed the meeting getting the sense of the room.
. Hum if support the BOF: very mild hum . Hum if oppose the BOF:
slightly louder hum
The majority of the room stays silent.
The room opposes the BOF.
|
